Not to mention that it's got all sort of bugs and traffic discontinuities. I just saw one of those "road train" things (a batch?!) on the central loop "crash" at the "Layer Counter Arch" while other cars just drove straight thru it.

The "Output Plaza" doesn't seem very outputtish - cars/embeddings seem to loop around back into the input.

Not sure why the Attention Plaza is occasionally shooting stuff down the storm drains/whatever in the center!

The sampler as centrifuge is a strange visualization!

So I guess the takeaway is that LLM are related to cars? Or maybe highways. Was this designed by Schmidhuber?

You're absolutely right!

Congratulations you're now a senior AI engineer

Just softmax it over the KV cache dude!