The limitation lies in Transformer architecture itself, therefore AGI was never a possibiblity.
It's wonderful, but not miraculous; And that's okay.
At this point big techs are just milking the hype of what already reached it's boundaries.
I dunno I'm confident enough to say transformer based architectures have really reached their boundaries quite yet but I would agree the hype milks a bit past where it seems transformers alone will take things.