Hacker News Reader

Comment by nemomarx | Hacker News Reader

There's different needs in tension I guess - customers want it to remember names and little details about them to avoid retyping context, but context poisons the model over time.

I wonder if you could explicitly save some details to be added into the prompt instead?

View on HN

I've seen approaches like this involving "memory" by various means, with its contents compactly injected into context per-prompt, rather than trying to maintain an entire context longterm. One recent example that made the HN frontpage, with the "memory" feature based iirc on a SQLite database which the model may or may not be allowed to update directly: https://news.ycombinator.com/item?id=43681287

Those become "options," and you can do that now. You can say things like: give me brief output, preferring concise answers, and no emoji. Then, if you prompt it to tell you your set options, it will list back those settings.

You could probably add one like: "Begin each prompt response with _______" and it would probably respect that option.

I wonder if it would be helpful to be able to optionally view the full injected context so you could see what it is being prompted with behind the scenes. I think a little bit of the "man behind the curtain" would be quite deflating.

Tinfoil's chat lets you do that, add a bit of context to every new chat. It's fully private, to boot, it's the service I use, these are Open Source models like DeepSeek, Llama and Mistral that they host.

https://tinfoil.sh/

Reply on news.ycombinator.com