Let’s say you could get 10x speed up with 10k and 11x speed up with 100k. The 100k option still wouldn’t make sense.

It’s a contrived example, but to me it’s not improbable that a lot of the spend comes from tokens that are used very inefficiently. After all developers were pushed to try using LLMs without looking at costs.

I’m not saying the others are wrong, just that you can only tell something about marginal cost effectiveness, not total cost effectiveness of LLMs by measures to reduce AI spending