- 1 week before releasing new model, they nerf the old. Probably to reserve compute for the new release. (insert your evil corp motive here). This is the third time I have seen this now.
- They run A/B tests continually. Hence the "how is Claude doing: good/okay/bad" question.
- They try to shrink the model while keeping performance, using quants, etc. they keep the name though. Opus at the first, is often not the same Opus a month later.
- My friend works at a top 100 spender on Anthropic. He complained about nerfing to his point-of-contact at Anthropic, and they did some setting change and it didn't happen anymore.
- He, European, says the models can get worse around 4pm, which is when America gets to work