Many theories of economic growth say that it is fundamentally generated by new ideas. When you have fewer people, you have fewer new ideas. So innovation might slow: in some models it even reverses.
How are skills passed two generations into the future? We would have to start over.
They are generated by highly educated people with genetic ability, who also have the means for a stable adulthood and at least a modicum of free time to develop their curiosities.
Not to mention the means to go into science. Very frequently another industry will pay more, and that person never becomes a scientist/researcher.