This is a fable. Many people reached old age in the empire and even the middle ages. A low "Life expectancy" doesnt mean people didnt grow old. It just means that many people die prematurely of (now) preventable things.
Death of the mother in childbirth ( 1 in 10) Death as a child because of unsafe surroundings Death as a child because of childlabour Death because of infectious wounds Death because of plagues and sickness Death because of random violence Death because of war Death by famine Death by accidents like fire in the house, occupational hazzard or hunting. Or the sinking of a ship.
All these things lower Life expectancy. But with luck and a good physique many of them could be avoided.