I would have appreciated if they had released the full distribution of impact though.
They don't give an upper bound though. 30% had at least one message degraded. Some proportion of that 30% (maybe most of them?) had some larger proportion of their messages (maybe most of them?) degraded. That matters, and presumably the reason we're not given those numbers is that they're bad.
Regardless of whether it’s to save money, it’s purposefully inaccurate:
“When Claude generates text, it calculates probabilities for each possible next word, then randomly chooses a sample from this probability distribution.”
I think the reason for this is that if you were to always choose the highest probable next word, you may actually always end up with the wrong answer and/or get stuck in a loop.
They could sandbag their quality or rate limit, and I know they will rate limit because I’ve seen it. But, this is a race. It’s not like Microsoft being able to take in the money for years because people will keep buying Windows. AI companies can try to offer cheap service to government and college students, but brand loyalty is less important than selecting the smarter AI to help you.
No, it's just the definition of sampling at non-zero temperature. You can set T=0 to always get the most likely token. Temperature trades of consistency for variety. You can set T to zero in the API, I assume the defaults for Claude code and their chat are nonzero.
You are absolutely right! Greedy decoding does exactly that for longer seqs: https://huggingface.co/docs/transformers/generation_strategi...
Interestingly DeepSeek recommends a temperature of 0 for math/coding, effectively greedy.
How many users forget they have a sub? How many get a sub through work and don't use it often?
I'd bet a large number tbh based on other subscription services.
I typically read corporate posts as cynically as possible, since it's so common to word things in any way to make the company look better.
Glad to see an outlier!
Unless you consider service responsiveness as a factor of integrity. Still waiting on a service message reply from third week of May. I’m sure it’s right around the corner though.