A test being a good indicator of human learning progress and ability is almost completely orthogonal to it being a good indicator for AI learning process and ability.

In their everyday jobs, barely anyone uses even 5% of the knowledge and skills they were ever tested for. Even that's a better (but still very bad) reason to abolish tests.

What matters is the amount of jobs that can be automated and replaced. We shall see. Many people have found LLMs useful in their work, it will be even more in the future.

IMO, it's a good opportunity to re-think about exam and future of education. For many schools, education = good results in exams. Now GPT-4 is going to slam them and say what's the point now!