While I'm really impressed with ChatGPT, and am one of the people who regards it as meeting my prior definition of AGI[0], I can still see its current flaws, and do wonder if this is a similar case, where the first 90% needed a major breakthrough but once that was invented anyone could do it and many wanted in on the economic opportunity… but the second 90% turned out to be just as hard, and so was the third, … and you need at least six nines[1] to really replace humans in these roles.
[0] All three letters mean different things to different people. To me: it's artificial, it's much too general to count as a narrow AI, and I count it as intelligent because the things it can do were the things I grew up thinking were signs of intelligence, like speak Latin, do algebra, and answer trivia questions, and also things I added later like 'write code' and 'pass medical and law exams'; even though it gets the answers wrong sometimes, I don't think my standard was ever "must be perfect" because nobody ever scored 100% on exams at school either.
At its best (and it's weird that it even has a best and a worst), the free version of ChatGPT has given me better code than one specific real human I've had to work with, more if you also add in the students. (And at its worst, it gives me stuff that doesn't compile and wouldn't do what I asked even if I fixed the compiler errors).
[1] Assuming a driver is making 1 decision per second that has a serious wrong answer, it would take eight nines to have just under one serious accident in a lifetime of 1-hour-each-way commutes, 5 days a week, 50 weeks a year for 40 years. I suspect the actual time between opportunities for serious mistakes is less than that, but probably at least once per minute even on an empty road.
For an LLM, I don't know exactly how good they'll really need to be, all I can guess at is that they're not going to have more than one opportunity to seriously mess up per token.