Good quality dairy is associated with high insulin-like growth factor 1 (IGF-1) levels in blood, which in turn leads to rapid cell growth. This explains why e.g. Dutch people went from being the shortest to the tallest in Europe.
He always pointed at food, that the generations after him had access to better stuff and more milk. The shortest member of my family is my niece who is totally anti-milk, but that's just anecdata.
So it might be some other macro-factor influencing growth, with milk being the most glaringly visible item on the table. But it does kind of make an intuitive sense, and the dutch are freakishly tall while they have some of the best dairy products in the world
Apparently, aside from better food, natural selection also had an effect [1].
[1] https://royalsocietypublishing.org/doi/full/10.1098/rspb.201...