But by having more people, you also have more people having more problems. In other word, the person who die early might not even exist at that year if the population was less. Instead of having an x population over y time span, you have an x/n population that span n*y period.

My point was that you need to have some kind of metric (suffering per human per year?), or otherwise the "quality of life" across different population size and time span doesn't make sense.