Hacker News Reader

Comment by YetAnotherNick | Hacker News Reader

Better comparison would be llama 3 70b whose pricing is $0.59/$0.79[1]. Gemini Flash pricing is $0.35/$1.05. For similar quality and price I would select open model, if only for the certainty that the model would perform exactly the same and can't be tweaked/deleted without warning.

[1]: https://wow.groq.com/ [2]: https://ai.google.dev/pricing

View on HN

I found myself making these types of comparisons over and over again to keep up with new models and pricing changes, so I made a Google Sheet for it with some charts to visualize outliers and trends in terms of model capability, throughput, and token cost. It's sorted by the Artificial Analysis Index [1], which incorporates both static benchmarks (MMLU) and dynamic crowdsourced elo (chatbot arena). Hope someone finds it useful:

https://docs.google.com/spreadsheets/d/1foc98Jtbi0-GUsNySddv...

Specific to Groq - the service is an incredible outlier for speed and cost, but even for quite small projects I've run into rate limits and similar errors. Great option for something like a bring-your-own-keys personal chatbot, but I wouldn't build a service on it. Groq Llama 3 70B and Groq Mixtral 8x7B are in the sheet for comparison but omitted from charts.

[1]: https://artificialanalysis.ai/leaderboards/providers

This is great - thanks!

groq is super fast and cheap, but it seems to have (for normal plans) a very-low 30 requests/minute rate limit; to get a higher-limit, it's "contact us" enterprise pricing (which I haven't done, so I'm not sure what the results will be).

edit: ah, currently it's "free beta access with low rate limits" and "enterprise" that is available, but they are planning to release "on demand pay per token" with higher rate limits soon; exact pricing unknown.

https://pasteboard.co/OnLHjusiv6QT.png

another limitation of groq, which may or may not matter for your use case, is there's no way to pre-fill part of the llm response, which can be a very useful technique (there's also no support for the more general feature of "restrict sampling to a grammar", tho that is a much bigger ask).

Reply on news.ycombinator.com