Never ending growth is an axiom because population growth is a given. Unless we forcibly control birth rates, that won't go away.
It doesn't logically follow that anyone controls birth rates by force, you don't need to invoke authoritarianism. Increased death rates/lower life expectancy and infant mortality can and would accomplish the same, entirely with no external driver. But statistics really bury the lede there, since the mechanism for those would be mass starvation, war, and decreased standards of living.