Ask HN: Is anybody selling API access to HuggingFace models? | Hacker News Reader

Ask HN: Is anybody selling API access to HuggingFace models?

3 pointsJuanPosadas7 comments

I've used the OpenAI and GCP chat-bison APIs with great success, but I'd like to put my money into more ethical companies with free-as-in-freedom-er models.

I don't want to fickle around with buying or renting GPUs or installing random LLM scripts that don't work 90% of the time.

I also don't want to pay $XX/month subscriptions, I want clean API billing like OpenAI and GCP.

What exists? Is anybody doing anything?

View on HN

This is not really an answer, but I really like the "AI horde" API setup.

https://aihorde.net/

Namely, you have a pool of gen AI volunteer workers constantly chewing through API requests. Host requests are prioritized, but the "downtime" of your AI worker instance is spent fulfilling other requests, which earns karma that can be used to prioritize requests from other hosts should you need more throughput.

Economically, this is way more efficient than self hosting or "renting" an instance that will be idle between requests, and you aren't paying a massive tax for a proprietary API.

Sort of related, I'm surprised there are no "edge" GPUs available in the style of cloudflare workers or AWS lambda. I heard that cloudflare was going to do something like this but it never materialized. It would be great (as an alternative to API access) to be able to pay for a worker GPU that you only used when doing inference. I know there are technical challenges to that, I wonder of it's still too hard to make something responsive enough to be useful?

What about replicate.com and RunPod?

Thanks! I was looking for something like this last year and couldn't find anything I'm glad to see progress in this space!

Many of the "free as in freedom" models are not trained or licensed for commercial use. For example, selling access to a LLaMA API would be a direct violation of the license the model was provided with.

I don't think you mean free as in freedom. Maybe freemium, I can't think of a closer short description. They don't cost anything but you're not allowed to use them for most stuff either.

Otoh, there are plenty of Apache and MIT licensed models. For example Open LLaMA and Falcon, Dolly, etc. HF lets you filter by license.

You're right. In fairness, the actual code for training LLaMA is "FOSS" technically. It's just that the model license (and weights afaik) are noncommercially licensed.

I guess it would be cool to see a paid Falcon/Dolly API, but in my experience those models get smoked by a good LLaMA 7b tuning. You would really have to hate OpenAI to seek out a non-GPT3 solution here...

Reply on news.ycombinator.com