Sam Altman said it himself. He seems like a reasonable source.
If you're familiar with other fields of AI, adding more and more layers to ResNet was the hotness for awhile, but the trick stopped working after awhile.
If you're familiar with other fields of AI, adding more and more layers to ResNet was the hotness for awhile, but the trick stopped working after awhile.
It is possible they've reached some 80/20 point and he is pretty honest about how much more extendable the current approach really is.
Would explain going to congress and asking for regulation (of their not-quite-there-yet competitors who they want a regulatory moat against).