There is absolutely no theoretical justification for this assertion that LLMs somehow have some emergent quantum mechanical behavior, metaphorical or otherwise.
In a formal Bayesian context, I’d call it “updating my posterior by adding data to the likelihood.”
And anyhow there's _plenty_ of theoretical justification for modeling things like this with various tools from quantum theory:
https://philpapers.org/rec/BUSQMO-2 https://link.springer.com/book/10.1007/978-3-642-05101-2
Also, I'm simplifying things a lot to make them accessible.
The OP goes into a lot more detail.
I highly recommend you read it.
>the superposition is unlikely to collapse to the luigi simulacrum because there is no behaviour which is likely for luigi but very unlikely for waluigi. Recall that the waluigi is pretending to be luigi! This is formally connected to the asymmetry of the Kullback-Leibler divergence.
The K-L divergence has absolutely zero discernible relevance here. The cross entropy loss function of a categorical predictor (like the token output of an LLM) can be formulated in terms of K-L divergence, but this has absolutely zero relevance to the macroscopic phenomena the author is conjecturing.
Forget “less wrong,” much of this article is not even wrong [0].
(wikipedia) "A simple interpretation of the KL divergence of P from Q is the expected excess surprise from using Q as a model when the actual distribution is P.
The sentence you quoted posits that whenever the LMM is in a state where it is "simulating" a nice and helpful person, the simulation is also consistent with an insane, violent person that's currently pretending to be nice, but not the other way around.
The author isn't talking about the loss or error of predicting individual tokens. If you look the larger scale behaviour, predicting, e.g., a scalar niceness value of the response based on one of two "modes" that you assume the LLM is currently in (either Waluigi or Luigi), then you'll be less surprised if Waluigi acts like Luigi than the other way around.
The probability distribution of niceness when assuming the LLM is in a state of "Luigi" would have a high mean and low variance, while the distribution for Waluigi would have a lower mean but a higher variance.
Thus, the KL divergence of Waluigi (that is, the probability distribution of niceness you'd predict when assuming the model is in Waluigi mode) from Luigi would be high, while the other way around `KL(Luigi, Waluigi)` would be low.
It should be easy to construct an example with concrete values using two normal probablility distributions.