I think part of the mechanism is that sickly hunter-gatherers die and do so early-ish, but agricultural societies can keep those people around.

Also if you have too many hunter gatherers around for a given area (and assuming they can't leave), some of them will die within perhaps a few years of starvation.

There's a much wider band of population densities for agricultural societies where they can produce enough calories to keep people from starving, but many will be miserable from malnutrition.

As an aside: in our over-populated hunter-gatherer scenario the average amount of food available might still be plenty, but there might be the occasional bottleneck during the harshest parts of winter.

Thanks to that bottleneck keeping numbers low, the typical day of life experience for your typical hunter-gatherer might be pretty nice. And thus you can get 70+ healthy years for many of them.