Hacker News Reader

Comment by JV00 | Hacker News Reader

You don’t need the entire codebase in context in every moment to migrate it. Also AI being non deterministic does not prevent it from one-shotting perfect solutions 100% of the time for simple enough problems. And every model generation brings this bar higher. So that’s really not a fundamental problem. And we can also implement llm inference deterministically if we want, it’s just that it’s not worth the loss in performance to do it.

View on HN

> And we can also implement llm inference deterministically if we want, it’s just that it’s not worth the loss in performance to do it.

My understanding is thats not possible (different from being practical), wonder if you have any literature, research to back up that claim?

This one from thinking machines: https://thinkingmachines.ai/blog/defeating-nondeterminism-in... Was discussed a lot also here on hn

Thanks for sharing, it was really good read. The authors in the article have done a good job for sure to validate their hypothesis and make it work for a particular subset of problem.

1. Local hardware, no networking or HTTP requests 2. No other parallel requests 3. 1000 runs bounded by 1000 tokens

I have worked a little bit in academia and I am not a big fan of the way favorable samples for the hypothesis are kept and unfavorable ones are thrown away. I might be wrong here, however its highly unlikely that the team would have just worked with one query. Chances are that a lot of different prompts with varying number of runs would have been tried to see what works and supports the hypothesis.

Reply on news.ycombinator.com