It's interesting that you faced this issue and came away with the exactly wrong conclusion. Economics is, at root, a physics problem. There's no way to get around that. Every aspect of the field comes down to asking questions about physical resources and their uses.

People can attempt to ignore the fact that economics is a special application of physics, but they can't escape the consequences of ignoring reality forever. The belief that it's all just conventions and cultural agreements is exactly like believing that the Moon is just a bedtime story.