Passing a 95% significance test doesn't have any connection to the notion that there is a 5% chance the drug is no good.
This is what you would expect from a blind application of a 95% confidence interval only if 50% of tested drugs were good.
This is what you would expect from a blind application of a 95% confidence interval only if 50% of tested drugs were good.
My understanding was that a 95% confidence interval implies the chance of a false positive of 5%.
From http://www.mit.edu/~6.s085/notes/lecture2.pdf
"In particular, we can ask for an interval around ˆp for any sample so that in 95% of samples, the true mean p will lie inside this interval. Such an interval is called a confidence interval."
You are likely to find the actual positive and 50 false ones.
However, that’s with a single test. If you run 5+ different tests you are likely able to distinguish the true positive.