I'm amused by the thought that the AI models are trained on human knowledge, but human knowledge doesn't contain a reliable method for determining what truth consists of, or what is true. I don't know how an AI could embody such a method itself.
Depends on how many connections a statement has. `1+1=2` has an extremely high number of connections, so it would be very hard to vary that to `1+1=3`. When you start to say things like `Daniupolomonotrofin activates the mitoyuicinain leading to weightloss`, very few connections, very easy to vary (lie).
But we have heuristic.
You do not even have the "perfect hammer", for example. But "close enough" is fine.
I want to improve the typography - the current feedback I'm getting so far is that the current design works well with two friends who have dyslexia, but I think that's more to do with the formatting than current typography... I have no good sense of typographical style though.
And now the more relevant comment!
Consensus truth can be far from objective truth as ideas don't compete based on value or truth or usefulness: but merely by how sticky, how replicatable in the brains of others.
https://en.wikipedia.org/wiki/Anosognosia
Right now, the AI is operating "inside the bubble" of our thoughts and so is unlikely to figure out our collective blind spot, but once one can interact with the world in more meaningful ways, we should pay really close attention to the mistakes it makes.