Hacker News Reader

So this work appears to work on inference speed and getting large models to work on simpler RAM constrained hardware, whereas my (private) work has been on getting simpler RAM constrained hardware to give agentic behaviour with small models. I've already started the work of using some of the learnings described in this repository. Looking forward to seeing what can be achieved.

View on HN

I'm also working on this thread in parallel because it has interesting business aspects, would be happy to see your progress when you release it

For me it's just about empowering people to jump on this LLM bandwagon with minimal resources. I'm not expecting top tier work, but I can already sent it off to check the weather on a certain day, or build a small Python GUI. There's still cases where it struggles and cannot recover - which is where I'm actively trying to improve it - without implementing some kind of case-by-case solution.

Reply on news.ycombinator.com