Hello everyone, we see a lot of LLM routing seen as just mapping difficulty to model tiers. This post explores the other half of the problem, that is when to switch models or stick to ones to account for cache breaks as well.
Would be happy to have feedback on this and discuss it.