Hacker News Reader

Looks great! Feature suggestion: would be great to plug in ollama for the AI parts. Not as great as a BYOK, but worth it to those who want to keep everything local

View on HN

Hey thanks! Good call. I had limited success getting Qwen 3.5 9B working for some of the longer prompts that require lots of json output. I feel like completely on-device is so close to being usable for this stuff though. I should revisit this, actually.

running locally is great, but I would wonder about the system requirements (I love my mac air but ollama on 8GB with an Intel i5 probably would be a little too much for the little thing). But having a toggle option would be great

Right? I feel like local model support is something that has strong ideological appeal and might influence someones feeling towards the app, but when it comes down to it most people will probably just use a cloud model for larger tasks unless they have beastly hardware. It's like how I have an Android in part because I might one day flash the ROM.

Ollama just exposes models via an OpenAI compatible endpoint though (I'm pretty sure), so adding that standard is probably a good idea. The prompts are a bit tuned for Gemini. I'd have to test how much that matters.

Reply on news.ycombinator.com