Hacker News Reader

Comment by torginus | Hacker News Reader

I don't get this whole thing. This sounds like a small model, that's very possible to run on the edge, and it seems like most of the things it does actually are useful there.

Why does this thing have to be another Web API?

View on HN

The extra funny thing about this is that the target audience is people who use agentic AI systems (system one thinking as a compliment to system two thinking). Setting up local decision / classification models is as easy as telling the agent you already use "set up laya on llama cpp locally https://github.com/ggml-org/llama.cpp https://huggingface.co/ggml-org/Laya-GGUF" (or your inference engine and model of choice, walk away for a minutes, and come back to it running on your computer.

The main benefit of Jev seems to be broader knowledge? But local models (especially the ones actually trained specifically for decisions, rather than ones that "support" it like embeddinggemma) are probably good enough for what most people use it for. They also seem to be quite easy to fine tune (I haven't tried, but seems to be what people say).

it still needs the full semantic "understanding" of natural language. So I don't see how a smaller model one-shotting a single token could be any better than other bigger models

> Setting up local decision / classification models is as easy as telling the agent you already use "set up laya on llama cpp locally https://github.com/ggml-org/llama.cpp https://huggingface.co/ggml-org/Laya-GGUF" (or your inference engine and model of choice, walk away for a minutes, and come back to it running on your computer.

I mean, yes, that works at dev time; but going from zero to nonzero local inference components makes an agent implementation suddenly much more complex to ship to production / drastically narrows the set of hosts such an implementation can be deployed onto.

Reply on news.ycombinator.com