People who are still afraid of dietary fat from the anti-fat craze of the 80s and 90s.
If you’re watching weight, cutting calories from milk of all places is the worst place to do it. At least from whole down to skim. All the healthy stuff actually just so happens to be in the tasty part, the cream, since milk does these thing of mixing the ”sugar” into the cream in the form of Lactose. Cutting that out just leaves you with a worst-tasting, less nutritious beverage aside from the fewer calories.
If you want fewer calories but still have nutrition, soy is the way to go.