Hacker News Reader

Can this be integrated with local LLMs or does it only support openAI?

Currently it only supports openAI. I'm looking into patterns for supporting local LLMs! I think the best approach for that might be to allow you to update the api endpoint to your own and assume you're using something that mirrors the structure of their API.

The main escape hatch for everything right now are custom nodes though. But then you'll need to bring your own templating pattern.

Ah, yea i definitely need local-only support.

My goal is to build a single binary, local only command<->LLM interface. Chidori looks to overlap with that significantly, but of course i'd need custom LLM resolution.

Reply on news.ycombinator.com