1. AI gets data about the patient and makes a diagnosis. This is NOT shown to doctor yet.
2. Doctor does their stuff, writes down their diagnosis. This diagnosis is locked down and versioned.
3. Doctor sees AI's diagnosis
4. Doctor can adjust their diagnosis, BUT the original stays in the system.
This way the AI stays as the assistant and won't affect the doctor's decision, but they can change their mind after getting the extra data.
6. Rankings are used to periodically "trim the fact" thus delivering more optimized cash flows to clinics that have been saddled with toxic debt
7. Sensing an opportunity AI providers start selling a $200 / month Data Leakage as a Service subscription to overworked physicians so that they can avoid the PE guillotine
I agree with GP's solution but we'd need regulation to prohibit what you describe.
Incompetent ones order unnecessary tests and exhaust treatment possibilities, which drives up cost billed to insurance.
Only the insurance industry and perhaps licensing bodies can pressure to keep the quality floor high, at least in terms of accurate diagnosis and prevention of overtreatment.
They need to write down their (initial) diagnosis before the AI answer is shown.
If #2 is correct and #4 wrong, the doctor has to answer for stuff.
It's trivial to analyse the pre/post AI involvement doctor diagnosis manually and see what's going on.
If a doctor is just putting "asdljasdaskjd" on the initial to unlock the AI answer, they should be promptly fired.