TIL Donald Knuth operates a bit like GPT-3 but for research paper narrative.
I've made it a habit to ask myself what I expect the output to be for any programming operation, and why. It forces me to gain clarity into my mental model of what's happening, and it immediately highlights deficiencies in my model when it's proven wrong.
I ask the same of others when I pair program with juniors or interviewees. I find super useful all around.
It is still statistical. You still want to predict a distribution over expected next token.
However, the function needed to estimate the likely next tokens are not simple max over enumerated next tokens like in a language model. It's more like a transition model in reinforcement learning.
The fact that a process can be described statistically does not make the process itself statistical.