You have a valid point; killing an animal in order to eat it is not immoral, assuming you don't let anything go to waste. Factory farming, on the other hand, has nothing to do with how nature works. It's all about who raises the animals and how.

It' the same thing with the environmental aspect. If the animals graze all their lives then it might even have a positive effect on CO2 uptake in the long run, and it definitely is required for some biomes to survive. But, again, factory farming isn't exactly natural.

And yes, that applies to vegan food production as well.