Hacker News Reader

sillyingon Detecting when LLMs are uncertain

I have a simple question. Suppose that to answer a question I can use different phrases, I know the answer but I have several ways to express it. Then a LLM in this case produces tokens with high or low entropy?

Edited several times: I think to avoid this problem the answer of the LLM should be constrained in expression (say Yes or No, fill the blanks, etc). I think in that case we would have a decreasing sequence of the entropy for next token predictions.

sillyingon Writing a Mathematica Interpreter in Typescript

Mathematica has thousand of functions, this seems to be just an algorithm for defining and using rules like that in mathematica.

sillyingon Why do random forests work? They are self-regularizing adaptive smoothers

So it seems that when you have different sources of errors the average of them cancel the noise. I think some property about the sources of error is necessary so in some sense the sources should be independent.