False dilemma. People don't need dairy milk (a lot of the world is dairy intolerant anyway) and there are alternatives that don't pollute anywhere as much, don't need as much land nor water and which are inexpensive and getting cheaper all the time.
Anyway, where does this "people don't need X" stop, though? You also don't need oat milk, tofu, soy, peas, etc. to survive.
Let's just go back to foraging for roots and berries and see what population size we can sustain!