Claude 3.5 Sonnet represents a step change in AI coding. It is incredibly industrious, diligent and hard working. Unexpectedly, Sonnet’s amazing work ethic caused it to regularly hit the 4k output token limit -- stoping its coding in mid-stream.
Allowing Sonnet to return code in multiple 4k token responses jumped its score on aider’s refactoring benchmark from 55.1% up to 64.0%.