Pretty simple without training the one shot the model never guessed even under training the pepperoni answer
I mean, it rediscovers it because it stored it hidden from our view. Use whatever words you like though.
There’s not enough space to store things like this.
If you have a 100 things to store, but only 5 slots, whatever mechanism you use to try and rediscover those 100 things using the 5 slots is not storing much.
The Q&A are purposely not guessable. The only way it could answer that correctly is it must have it baked into the weights somewhere. What I'm saying is that the configuration of the weights hid that information then random chance reemerged that information again after it had forgotten and self reinforced it without any training signal to do so.
> random chance
Ah there is the rub, it’s not random. It’s biasing it a tiny bit.
Let me try to come up with another analogy. Suppose you’re flying a plane into Siberia and start out in a specific direction. And I want you to fly over a specific town.
Storing the information would be giving you the exact coordinates of the town and instructions to reach it. What happens here is closer to bumping into the plane with a tiny rock so the trajectory is moved in a direction where you naturally see this town on the new route.
This was pretty much a 2D analogy, can you imagine a 1024 dimensional version of this? That gets pretty close to what happened.
It's something like that for sure, but we don't really have the theory yet, not even the jargon for it.