Hacker News Reader

one of the points is that LLMs are NOT deterministic so you can't run them yourself a thousand times and expect the same output.

View on HN

That's true, but if the LLM is calling a tool to provide the base information to extrapolate, the results will not change even if the sentence structure is never the same.

Debian has an extensive toolchain to check things. Having a set of deterministic tools which feed LLM with data can be beneficial, though.

Again, I'm not defending the LLM here, just want to show how hard the problem is to begin with.

And LLM "summarizes" the tool output as a completely unrelated and at times self-contradicting text.

If there are deterministic tools involved - just return their output as is. It's usually quite readable if needs a bit of adjusting

When was the last time you used a LLM? This is not an issue with current models.

Reply on news.ycombinator.com