They don't really outperform human experts on real tasks yet (no matter what some superGLUE or other benchmark shows); but in general, once a system can solve a particualr task well, it would be plausible to outperform human experts simply by not making random errors.

If we have multiple human experts annotate a NLP task and measure inter-annotator agreement, it will be far from 100%; part of that will be genuine disagreements or fuzzy gray area, but part of the identified differences will be simply obviously wrong answers given by the experts - everyone makes mistakes. The same applies for many other domains - business process automation, data entry, etc; no employee will produce error-free output in a manual process, no matter how simple and unambiguous the task is.

And for simpler tasks the computer can easily make less mistakes than a human - especially if you measure the human reliability not for a few minutes of focus, but for a whole tedious working day.

Maybe the gibberish GPT-3 spits out is actually true, and our puny monke brains are just too weak to understand it.

See, if you were an AI, you would understand EXACTLY what the poster means by this.

Parent probably meant that it outperformed a human expert in some specific task in the area of natural language processing - for example the task of converting a spoken language into a written language...

TBH, I believe we need another AI winter to come up with something better than the AI "black boxes" we have right now.