Hacker News Reader

Comment by bangaladore | Hacker News Reader

You can use something like OpenRouter, which lets you access essentially all commercially available models. Including open-source models. There are no rate limits.

You pay a different rate per model (OpenRouter shows the pricing transparently). You load your account with credits. I use it daily (undoubtedly far more than the average user) and loaded 50$ with credits five months ago, but I still have over 1/2 of it left.

I think it is hard to believe that Kagi would be any cheaper and have no rate limits.

View on HN

Keep in mind that, while OpenRouter gives you the upstream price for OpenAI/Anthropic models (so you pay the same per token), there's a loading charge, so if you want to load $10 in credits you pay $12 or so.

This means that it's more expensive than calling OpenAI directly, even though they have the same price per token.

Where the loading charge is amortized over all the calls made.

If you want to use precisely one API, paying directly for that API is cheaper. However, that's only true with closed-source providers. Anyone can host a server running llama 3.1 that OpenRouter could (in theory) use, bringing price competition to model cost. Closed-source models have a monopoly and can set their price wherever they want.

I'm okay with spending an extra 2$ every six months to access the APIs of any model I want.

Sure, but I only use the hosted APIs, so for me it doesn't make much sense to pay the extra premium. Maybe it doesn't for others either.

Just plug your API keys into a front end like https://github.com/enricoros/big-AGI and pay as you go for all commercially available models

Have you tried open-webui?[1] I've been using that and really loving it, but wondering if I should try out big-AGI

[1]: https://github.com/open-webui/open-webui

Not to add more to the mix, but I've been very happy with Librechat: https://github.com/danny-avila/LibreChat

LibreChat is also one of the few LLM chat interfaces that works with both external APIs (OpenAI, Anthropic, etc) and local ones (hosted via Ollama, etc) out of the box.

If it has a downside, I'd say its a little more involved to get setup, lots of docker containers, etc then a more batteries-included approach like Jan.

https://github.com/janhq/jan

There's also BigAgi (really a weird ass name - probably hurting them) that is good for the same use case. Just paste in your API key and you get a really nice UI to chat through at-cost.

https://get.big-agi.com/

A nice thing about this as I'm reading it is you can hook up OpenRouter to it. OpenRouter's interface leaves a lot to be desired.

Chatblade cli is also worth checking out. No loading fees and you can pipe code results to files.

Thanks for the hint, a usage based model looks way more attractive to me right now.

deepinfra is pretty easy to use too, for llamas and other "open" ones.

OpenRouter absolutely does have rate limits:

https://openrouter.ai/docs/limits

...haven't had issues with them, but they are there

Good catch. It's not relevant if used as an "assistant," though.

And if you cared and loaded your credits, you can do 200 req/s (maybe also a surge limit? docs are unclear)

Reply on news.ycombinator.com