He is not talking about a cure that works and that no one understands, he is talking about an AI making its way out of the trial just to get to phase 3...
If AI found a way to exploit clinical trial design, it would be noticed and the errors corrected. In fact, that would be a major win because it would probably lead to improved clinical trial designs.
Do you think so? So far, AI is gaming lots of things, and I don't see much fixing in the processes.
I suppose if we closed all the loops and the entire end-to-end process was completely automated, it wouldn't be surprising if we failed to notice.
If we followed the precautionary principle and waited until we understood everything there is to know about every drug on the shelf, a lot of people now living would be dead.