This is what I don’t understand. Supposedly LLMs are trained on human text. Why do they come up with such unrealistic prose? Is it intentional because the companies want the tells to be obvious?
Getting it to write well is really hard because there’s no real way to verify whether it’s good prose or not. You and I can tell, but we can’t write a verifier that codifies our judgment.
Maybe they’ll find a way to improve this, but for now it’s certainly one of the harder problems to solve for LLMs.
Part of it is that I think they also have poor theory of mind, which I imagine is also a hard thing to train it to do.
In any event, other LLMs may not automatically have the problem, and don't even require such a prompt. This is a Claude problem.
LLMs are trained on an extraordinarily diverse range of human expression, yet their default behaviour collapses much of that diversity into a surprisingly narrow conversational register. This produces a pathologically median conversational style. The result is a remarkably consistent tendency towards the middle, which works shockingly well 99% of the time. But in the remaining 1%, when the median isn't a position many actual humans would occupy, the artificiality becomes glaringly obvious.