> So far it seems to me that self-attention really brought new capabilities to a network
Do we have a layman explanation for what makes self-attention so uniquely powerful? Something more than "it lets you do self-attention".
Do we have a layman explanation for what makes self-attention so uniquely powerful? Something more than "it lets you do self-attention".
So (modulo a _lot_ of details) it increases the power from that of a "calculator" to that of a "computer".