yeah I found this one extremely hard to explain to non-technical people.
181 karma · joined May 29, 2011
Intuitively I understand this due to how context windows work and you're looking to increase cache hits, has Anthropic tried compact/summarise on idle as a configurable option? Seems to have decent tradeoffs + education in a setting.