We found a Neuron in a Neural Network

Nothing new to see here. They pinpoint the nodes where the training bumped up the numbers for one token while not firing other tokens.

Yes, I’m being a bit reductionist, but GPT -> transformer architecture -> neural network. We just have more detailed techniques, a lot more data, storage, and processing power now. But the basics of how a NN works hasn’t changed.