The transformer architecture is arguably the most important breakthrough in NLP, and language is the predominant mode of communication between humans, so I fail to see how its "just a load of hype"
Could you name a bigger breakthrough in AI?
Could you name a bigger breakthrough in AI?
Maybe even convolutional neural networks, because they showed that ANNs are viable and are what really got the ball rolling.
[1] K. Fukushima, Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position.
[2] F. Rosenblatt, The Perceptron: A Probabilistic Model For Information Storage And Organization in the Brain.