> LLMs are designed to create “helpful” and “convincing responses” without any overriding guarantees regarding their accuracy or alignment with fact.
This is an incoherent goal: accuracy and alignment with fact are a big part of being convincing and a bigger part of being helpful.