Less food miles. Less cheap availability of dairy will encourage movement to less polluting alternatives. Less intensive farming leaches less nitrogen.
Anyway, where does this "people don't need X" stop, though? You also don't need oat milk, tofu, soy, peas, etc. to survive.
Let's just go back to foraging for roots and berries and see what population size we can sustain!