Hacker News Reader

You can set a custom URL to use Ollama which is OpenAI API compatible. Llama3 8b runs quite fast for me on a M1 Max. I'll be making use of this feature I think (haven't tried it yet).

View on HN

No, unfortunately it's not possible to set a custom URL... OpenAI is either hard coded or using a configuration option that isn't exposed.

Advanced settings -> "URL for AI API"

It is exposed from Advanced tab under "URL for AI API". You do have to change the model name from the preselected and offered OpenAI GPT model names. Otherwise this AI API URL isn't used. Haven't yet tested with local OAI compatible model yet, though, but did manage to break it using bad address :)

How do you change the model name from the preselected and offered OpenAI GPT model names? I don't see the option to do so.

click the name instead of the dropdown arrow

Note that to make use of ollama, you'll have to wait for the next release or upgrade to a beta release --- while you can change the API right now it's using a newer format for the payload, that's fixed in the beta/next release.

Reply on news.ycombinator.com