I think this article pushes the premise farther than is reasonable.
The best anyone expects from an LLM "truth vector" is that it would encode the model's belief about whether the statement is true. Of course a perfect truth oracle is impossible.
The best anyone expects from an LLM "truth vector" is that it would encode the model's belief about whether the statement is true. Of course a perfect truth oracle is impossible.
Whatever vector we choose, there always would be contradictions.
And while people also have internal conflicts, they are capable of adapting their belief system, while adjusting a vector would just give an LLM new self-contradictory classifier.
But LLMs are not logical; they are statistical. It can only 'believe' a statement is true in the Bayesian sense, where the statement agrees with the priors.
The latent space encoded in the weights of a model is itself an object of study, just as interesting as the model itself.
There's definitely people around that have no idea about the "soft" sciences and how an LLM while constructed from a "hard" science is almost entirely a "soft" science object.
Thus said HAL9000
You don't represent all of us buddy. Some of us love truth above all else.
So? Some people get off on pain, doesn't make the statement "People try to avoid pain" false.
I think even that's too-optimistic: The LLM is a document-extender, so its "belief" is whether a token seems like it would statistically fit-next in a partial document, based on prior documents. This is usually not the kind of analytic truth we're interested in, and we've already figured out how to constantly extract it.
If we peek at vectors and weights, we'll we'll probably end up measuring the moods and styles for whatever tokens are about to get emitted next, whether that's dialogue for a fictional character (of various kinds), a narrator, or an impersonal memo conclusion paragraph. We'll be measuring "earnestness and conviction", on the same level as "loquaciousness" or "pleading" or "talking like a pirate."
So is Truthiness [0] what we really want? Probably not. If our document described the character as Yoda, then The Force connecting all existence ends up truthy. Using "a really gullible person" can repeat anything you supply as truthy. Even if we set things up as "a respected encyclopedia article" or "a relentlessly logical super-genius", we're really changing the influence mix of styles and biases, rather than creating a logical mind independent of text inside the LLM.
A drop of water falls on your hand. Is it raining? Depends. Are you painting a watercolor picture outside? Then probably yes. Or are you going to the store? Then probably no. So truth is inseparable in practice from usecase. Is a whale a fish? I don't know, are you a geneticist or a poet? It's all mood and usecase. I'm not convinced there's any difference in kind between the LLM's speaker-selection and a human's choice of research field.
That is to say, it's not that I think you're wrong, it's that there is no other pursuit of truth than what you describe. If anything, the LLM's pursuit of next-token prediction is unusually honest for a truth-seeker.