Google literally publish the attention paper. Have people not been paying attention? Google has been the only company I’ve been watching that really understands what they are doing.
There are serious philosophical problems with betting big on an interesting outcome like ChatGPT, even though it seems obvious (Google also did this of course), but creating the best architecture to do that job seems like a first-principles intelligent move, because there was no reason to keep using graphics cards except that they "did the job."