I always thought that was more of a conspiracy that milk helps you to grow. My mum always tried to get me with that but I always didn't give a damn because I thought it's not true anyway.
He always pointed at food, that the generations after him had access to better stuff and more milk. The shortest member of my family is my niece who is totally anti-milk, but that's just anecdata.
So it might be some other macro-factor influencing growth, with milk being the most glaringly visible item on the table. But it does kind of make an intuitive sense, and the dutch are freakishly tall while they have some of the best dairy products in the world
Apparently, aside from better food, natural selection also had an effect [1].
[1] https://royalsocietypublishing.org/doi/full/10.1098/rspb.201...