Hacker News Reader

We’re trying to do this at https://openrouter.ai

https://gradient.ai/ is doing that with llama2

Looks really promising. I wonder if the similar pricing to OpenAI means that Gradient is also(?) bleeding money even if they get a good customer base. Or are these prices sustainable over time?

Good question, esp as Gradient fine-tuning is so much cheaper than Open AI's

Yeah it's even cheaper. Although it looks like it's about the same in proportion to approx model size/expected quality? They haven't launched any >13B model yet, although they plan to.

I'm not into open source LLMs in the slightest, and yet even I've trivially found tools to do what both you and the poster above you wanted

lmstudio actually does what both of you want: provides an easy GUI and serves up your model over a local endpoint that mirrors the OpenAI API.

There's just too much noise in terms of the tooling for LLMs, the solution is fewer higher quality solutions, not more solutions

Oobabooga exposes an API.