Hacker News Reader

Comment by vengadanathan | Hacker News Reader

No necessarily, we are getting similar result gemma 4 12b, predicting one output token for label. it is just prefill and with nvfp4 caliberated , we are getting 70-80 ms p90 for the classification workload we have.

not sure what made you think you cant go cheaper than jev. it is being done for a long time.

View on HN

Can you host files cheaper than Dropbox? Of course you can. :)

Well if there is compliance / data privacy need to it, it makes absolute sense. We operate in vocie ai industry and run in regulated environment. Jev etc doesnt support languages we want (indic languages) and for our usecase and it is not self hosted either, so running fine tuned LLM based classifier is more accurate and low latent (no external network hop as it runs in VPC). Idea is to re-use as much as possible but if compliance, accuracy becomes bottle neck it makes more sense to do it yourself.

Right.

Once you discover what the useful problem to solve is and how to utilize it in production, you can of course replicate it.

This is not alchemy taught by some wizard in secret.

Very often, the innovation is in identifying what to build, what has interesting use cases, what people will pay for.

Once this is established, there is further research optimizing it further, replicating it locally, etc.

The fact that anyone could have come up with it is irrelevant.

Reply on news.ycombinator.com