Confidence scoring is fine when you provide the right context, like everything else LLM related. I use grading scripts for things that can be measured directly so the LLM just operates on it. For things less well-defined, I use a grading matrix with explicit criteria on what constitutes an A to F grade, usually with an assumption of an A grade that drops depending on specific findings.

If you are asking about confidence and somehow it can’t be defined either mathematically or in a grading rubric, then you the implementer need to start asking yourself the questions, because then you got a target that you don’t even know how to measure. The LLM isn’t even the problem at this point because another human would also just be going off gut feeling and vibing a number that may or may not be accurate.