I mean just think about it: you're putting food that grows mainly by direct energy from the sun (which is already a lossy process) and processing it through animals before eating it (in the form of that animal's meat). This is by definition an inefficient process, and we'd recover something approaching the entirety of the American continent of land by making this switch.
Obviously this is hard to accomplish given how deeply ingrained meat-eating culture is.