GPT-4o mini is $0.15/1M input tokens, $0.60/1M output tokens. In comparison, Claude Haiku is $0.25/1M input tokens, $1.25/1M output tokens.
There's no way this price-race-to-the-bottom is sustainable.
There's no way this price-race-to-the-bottom is sustainable.
Think about it this way: Imagine if every email you sent or every online forum post you commented on provided incentive for the provider.
Venture-backed companies can lose money for years. Sometimes it pays off in the end, but making predictions about profitability seems hard inside a bubble.
Also, some industries like manufacturing solar panels have high market growth but they’re unprofitable for most manufacturers.
So I think it remains to be seen if OpenAI knows what they’re doing. It doesn’t seem like the sort of thing armchair arguments are good at predicting.
If they're running at a loss, it's a great way to take shots at the competition and especially with the added advantage of model capability.
Get more customers onboard, play around with margins as required.
If you take a loss on every sale, it is impossible to make up for it with volume. The result will be a loss magnified by the volume.
Just be careful when they start building the walls. And they will build those walls.
To earn appreciable revenue at this price, an LLM company needs to be regularly generating multiple internets worth of text.
On the one hand, generating multiple internets of text seems outlandish.
But on the other hand, we're now approaching the point where you can start building LLMs into software without fretting about cost. Now that you can buy ~30 pages for a penny (instead of a dollar) you can really start to throw it into websites, games, search bars, natural language interfaces etc. without every user costing you much.
But small models are not the endgame for these AI companies, as truly general intelligence is a market worth trillions.
What this ~98% cost drop over 2 years hints at is that when AGI does arrive, it might not be horribly expensive.
- More prone to hallucinations
- Worse at citing sources for people to double check outputs
- Can't be updated without retraining
- Can't impose knowledge access controls for different users
Why not?
Algorithm (no, I don't mean Mamba etc, you can still use decoder-only transformers with some special attention layers) and engineering side there's still at least 10x improvement possible. Compared to what TensorRT-LLM is able to achieve now.
My concern is, this is only possible because of scale, so local LLMs are going to be dead in the water.