Most software engineers do not have a deep understanding of CPU architectures. In fact they probably don’t even have a shallow understanding and get around just fine. How many of them are looking up the instruction set for the CPUs they deploy their CRUD app to in EC2?
Basically every part of the original transformer was replaced with something more efficient or better:
LayerNorm -> RMSNorm
Sinusoidal position encoding -> RoPE
MHA -> GQA
ReLU -> GELU
What this means is that there is ample opportunity to improve on what we’ve done thus far.
Yes you might progress the field, but will you really understand why? You can make up an explanation and anthropomorphise it with a few contrived diagrams and everyone will cheer!
Linking to a Twitter thread about training a small LLM to be good at Wordle is not an example of what I'm talking about. It might well be a useful task but it doesn't allow us to understand deeply what's happening.
Now with llms we need way more that 6 dimensions so we can start thinking of assigning matrices to each 3d point for example. That allows us to increase the dimension from 3 to 3 + whatever the matrix dimension is.
We can visualize the matrices instead of having numbers as having colors for each entry, so they can be a sort of cube with each vowel being a different color.
Now you can start to visually intuit about how these massively high dimensional spaces can be formed of these colored matrices that can react to some input training data.
That's a start of an idea for intuiting things that might seem impossible to have an intuition about. I think visualization is a great way to start.
visualizign 100 dim matrix will not tell you how llm work. so what you even talking about.
Depending on your knowledge of math, I recommend starting with linear algebra, building an understanding of the equations and try to visualize more and more complex systems, then study llms to see how you can apply your linear algebra intuition to your understanding of llms.
VTK is a great toolkit for visualizing complex systems. 3 blue one brown on YouTube has other visuals that might help you.
It takes time but it's possible. Good luck.
> Visualizing large dimensional matrices is a way of starting to develop intuition about llms.
one last time before i disengage. how do you know this and what intuitions have you personally developed.
I've spent a great deal of time studying llms and NN architectures in general. This allows me to intuit things about them and that intuitive understanding for my comes mainly from visualizations.
Are you able to visualize much about llms? I ask because if not, starting by learning to visualize high dimensional complex systems is what I would personally recommend.
Check out this video for a really nice visualization into NN architecture https://youtu.be/QgH9sr7G13Q?si=iTn1FiFsYCZCJKE_
Like a cat who knows how to teleport so rapidly around the room that he becames a blur. All the while taking into account stuff falling around he knocked down.
As you say, there's a lot of value to unlock with understanding the generic principles, and I would add specific application.
Lots of people are building fantastic Tools or pulling down million dollar salaries without groking the precise representation of a single weight.
I'll borrow my response from a far greater applied mathematician than myself: https://www.youtube.com/watch?v=_oNgyUAEv0Q
But the load-bearing assumption is that intuition has to be human-shaped intuition. Humans can't intuit thousands of orthogonal directions because we project everything down into a 3D metaphor and hope it holds. That's a fact about our hardware, not about the systems.
And the reason why is the most interesting part: nothing requires the compression step. A model or an agent can operate over the actual objects, holding thousands of runs and ablations in context and noticing regularities in the native dimensionality, without translating them into a picture of a ball rolling down a hill. No bottleneck at "can you visualize it."
So the narrower claim: it's not that intuition here is impossible full-stop, it's that human intuition is unreliable. Your post-hoc rationalization point is evidence for that, not against it. The story exists because a person needs something to hold in their head. Drop that requirement and the failure mode goes with it.