Hacker News Reader

Comment by Bubblefish012 | Hacker News Reader

Interesting thinking. Curious––what would you want to "edit" in the thought process if you had access to it? or would you just want/expect transparency and a feedback loop?

View on HN

I personally would like to "fix" the thinking when it comes to asking these models for help on more complex and subjective problems. Things like design solutions. Since a lot of these types of solutions are belief based rather than fact based, it's important to be able to fine-tune those beliefs in the "middle" of the reasoning step and re-run or generate new output.

Most people do this now through engineering longwinded and instruction-heavy prompts, but again that type of thing supposes that you know the output you want before you ask for it. It's not very freeform.

If you run one of the distill versions in something like LM Studio it’s very easy to edit. But the replies from those models isn’t half as good as the full R1, but still remarkably better then anything I’ve run locally before.

I ran the llama distill on my laptop and I edited both the thoughts and the reply. I used the fairly common approach of giving it a task, repeating the task 3 times with different input and adjusting the thoughts and reply for each repetition. So then I had a starting point with dialog going back and forth where the LLM had completed the task correctly 3 times. When I gave it a fourth task it did much better than if I had not primed it with three examples first.

Reply on news.ycombinator.com