Initially I thought it's a bloody stupid idea, however at this stage I reckon we need it or a lot of boomers are going to be ones hotted into singing away all their wealth away.
69 karma · joined December 16, 2012
Initially I thought it's a bloody stupid idea, however at this stage I reckon we need it or a lot of boomers are going to be ones hotted into singing away all their wealth away.
now you can run an openai-compatible proxy for all of your "local model" needs.
currently using it as my custom provider in repoprompt
pkg includes: client, cli, and proxy
We built something like this for ourselves here -> https://www.npmjs.com/package/@kluai/gateway?activeTab=readm....
Documentation is a bit sparse but TL;DR - deploy it in a cloudflare worker and now you can access about 15 providers (the one that matter - OpenAI, Cohere, Azure, Bedrock, Gemini, etc) all with the same API without any issues.
Wondering if it's a weird PEBKAC or someone else has had the same experience?
Source: product of 2 former commie countries. The after effects probably take 50+ years to undo