Hacker News Reader

I would recommend looking at OpenRouter, if anyone is interested in implementing fallbacks across model providers. I've been using it in several projects, and the ability to swap across models without changing any implementation code/without managing multiple API keys has been incredibly nice:

https://openrouter.ai/docs/quickstart

View on HN

Have you measured how much latency it adds to each query? Naively I'd expect adding in an extra network hop to be a pretty big hit.

It's really pretty reasonable. I don't notice it at all. Nothing as bad as trying to relay 1min.ai or something

Anecdotally, there's been no obvious performance hit, but this is something I should test more thoroughly. I'm planning on running a benchmark across a couple of proxies this week--I'll post the results to HN, if anyone is curious

Reply on news.ycombinator.com