GRE writing: 5 for Claude vs 4 for GPT-4

Bar exam: 76.5 for Claude vs 75.7 for GPT-4

I think it's better at retaining memory with longer contexts, creative writing, and (when desired) longer responses.

As an example for creative writing: 2 is a slight degradation from 1.3 for this specific prompt, but Claude 2 is still better than any OpenAI model.

> Write a love letter. It needs to be written as if it was from Linus Torvalds, rudeness and all.

Context length is much higher than GPT-4

>Does this perform better than GPT-4 on any metric?

Nothing will. You need a dozen H100s just to run inference for GPT-4 [0]. The point is that smaller models can still be very useful.

[0] https://www.semianalysis.com/p/gpt-4-architecture-infrastruc...