For all the talk about fat content and quality, the topic of hormones in milk should be discussed. Cows load their milk up with hormones to stimulate growth in their calves. We’re not calves and don’t need stimulating. Further, modern dairy farms pump hormones into cows to induce higher milk production-thus more hormones make it into the milk in the container. There are even morphine-like compounds in milk and dairy products. They’re even more concentrated in cheeses. These same morphine compounds can be concentrated and given from a human mother to her baby through breast milk in non-trivial doses (though the conditions hopefully make this rare). Then there’s just the negatives of having additional cholesterol through the diet.
None of that’s going to be in plant milks.
Ref “your body in balance” by Barnard.