I would have thought that the confidence scores available from CNNs but not from LLMs _would_ be valuable.
Clinically, I'd imagine that choosing the right final diagnosis with relatively low certainty is often less useful than reporting the level of confidence (high or low) of each candidate diagnosis. A set of plausibles is surely better than a single wrong ID.
Clinical tests, especially when used by nonspecialists, are useful not only as diagnostic endpoints. Often they're essential to separate plausible diagnoses from implausible. If a CNN can rule out all outcomes but one with high certainty, that's nothing to sneeze at.
And if it could also _explain_ why the certainty of each candidate was high (or low) -- as non-LLMs often can do better than LLMs. That should have value too, especially in establishing an audit trail to defend the chosen diagnosis.