> The academic pyramid was never sustainable

Theoretically it wasn't, or every human on Earth would be a PhD candidate within the century.

However there was a period after WW2 where a lot of empty professor chairs needed to be filled and so I believe that people came to expect the growth. So for example you would have the post-war generation push their kids into it, saying "but of course you can be a professor like me, they will be beating down your door to get you hired!" But of course that seems to have stopped around the end of the century.