Hacker News Reader

It's an inherent problem with on-device AI processing, not just in the browser. I think this will only get better when operating systems start to preinstall models and provide an API that browser vendors can use as well.

Even then I think cloud hosted models will probably always be far better for most tasks.

View on HN

This specific problem is certainly not one for all on-devices AI processing. As someone else mentioned, there are unique UX and browser constraints that come from serving large compute intensive binary blobs through the browser (that are almost identically shared by games).

Separately, having to rely on preinstallation very likely means stagnating on overly sanitized poorly done official instruction-tunes. With the exception of mixtral7x8, the trend has been the community overtime arrives at finetunes which far eclipse official ones.

> I think cloud hosted models will probably always be far better for most tasks

It might depend on just how good you need it to be. There are lots of use-cases where an LLM like GPT 3.5 might be "good enough" such that a better model won't be so noticeable.

Cloud models will likely have the advantage of being more cutting-edge, but running "good enough" models locally will probably be more economical.

I agree. The economic advantages of a hybrid approach could be very significant.

Reply on news.ycombinator.com