Perhaps we’re starting to see the limits of the machine learning era in AI. We may have nothing more to teach it. There are a number of ways to go from here, none of which is entirely within our control or understanding.
GPTs have a small spark of higher level reasoning. Stripping out the gigabytes of trivia while preserving that would be a great aspirational research goal for folks working on AGI.
There are countless books that haven't been digitized.
The next evolution needs to be HFML.