Hacker News Reader

It's expensive to run it locally at full quality, but at least on my setup, its about 5 times faster than any API, and is completely private.

> trying to run it locally or with your own hardware is pointless.

Serving local models has advantages other than price. If you work in restricted industries, or have a strong need to protect your IP, or if you just value privacy more than cost, you now have options.

If you don't do any attention steering, custom decoding or meddle with the weights maybe. Services are worthless unless all you do is write positive prompts.

As others have mentioned, there's the privacy factor as well.

With the cost of electricity, hardware depreciation and tok/s it rarely makes sense to run locally.