That's the problem of reasoning on timescales of civilizations, yeah? You can't really pay attention to transient effects and still make meaningful policies.

For example, consider how many people died mining coal in the past couple of centuries, or how many natives died when colonizing forces spread them diseases or outright committed genocide.

Sure, it's awful, but would the world really be a better place if America were limited to some traders on the East coast? Or if England had never really gotten the Industrial Revolution thing figured out?

We can't really reason in human terms when talking about the species writ large.