This is interesting!
This is the first time in the "frontier LLM world" that I've seen a pricing split along a token boundary, in this case, less than or greater than 100k tokens...
In the past, first we saw ChatGPT, then we saw Claude, then we saw commercial frontier competitors, then we saw local open LLM's (Deepseek, etc.), then we saw "thinking" LLM's, i.e., use the LLM itself to generate a text plan to answer the user's prompt behind the scenes, then use that as a second, hidden prompt to actually answer the question, then we saw frontier pricing split across several different models (depending on how much behind the scenes "thinking" aka, "better hidden query plan text writing" that they did, and how many levels of this that they did) and these models may have had different context/token sizes but were not separately billed for prompt size, and now we're seeing pricing based on query size in tokens.
Now, I'm wondering how many levels or "pricing tiers" this could segment into...
In theory, an LLM provider could make 1k, 10k, 100k, 1M, 10M token context pricing tiers for the same model...
Would there be any practical benefit to doing so?
Well, that's a function of how much memory/compute are used by different sized prompts, how many prompt requests are incoming at any given point in time for a given commercial LLM provider, etc., etc.
I don't have all of those numbers.
In theory, an LLM provider could also charge less during nights or weekends -- times when electricity is cheaper, if those cost savings are passed on through the data center/compute supply chain, but again, I don't have all of those numbers...
Still it is highly interesting to watch product/service segmentation and differentiation in this highly competitive marketplace...
What will be the next innovation in terms of speed/efficiency/quality and/or price segmentation?
We don't know... but this LLM pricing split on a 100k token boundary is highly interesting for AI market watchers, and possibly for Economists as well, present and future!