Comment by Booktrope | Hacker News Reader

Comment by Booktrope | Hacker News Reader

Parent Full thread

I am troubled by the mammogram analysis. These sorts of health care examples (mammograms, AIDs testing, etc.) are often used to explain Baysian statistics and obviously make an interesting point. But as someone below points out, these tools are meant to be used in situations where we have incomplete information. We have direct experimental evidence of the accuracy of a positive mammogram in predicting cancer, and the accuracy is far, far higher than 7.8% From direct studies of patients who get positives, it looks like about 80% of them actually have breast cancer. See, for exmaple, http://www.cancerresearchuk.org/health-professional/cancer-s...

It may be that the author of this article simply go mixed up -- he reports than 1% of the population that 80% of the women who have cancer get positive mammograms, while from what I can see, evidence actually shows that 80% of the women who have positive mammograms have cancer. So the problem with the example might just be a matter of garbage in garbage out.

Mammograms, incidentally, are still controversial, especially doing them on an annual basis, because if you,for example, do a test with 20% false positives 10 times, you're pretty likely to get a false positive in at least one. But that probability is still not 92.2%, which is what the article suggests is the false positive rate for a single mammogram.

But it's disturbing to see statisticians flippantly saying things like, only 7.8% of positive mammograms represent actual cancer, when evidence shows, 80% of positive mammograms represent actual cancer. Survival rates for breast cancer have gone up and most everyone agrees early detection of cancer plays a role. I certainly hope that any woman who reads this article recognizes that, if she has a positive mammogram, the changes are much, much higher than 7.8% that she has cancer.

I've seen the same arguments made about AIDS tests. Why is it that statisticians like to use examples of life threatening illnesses and present a Baysian model that vastly underestimates the effectiveness of tests that could be crucial in saving lives?

On the other hand this paper: http://www.ncbi.nlm.nih.gov/pubmed/21249649 suggests that for every correct treatment there are 10 mistreatments and 200 initial misdiagnoses.

How could the numbers differ so much from study to study?

I think you're misreading that study. It compares the number of women estimated to have their life prolonged because of a mammogram, to the number of women estimated to have been treated unnecessarily because of a mammogram. The issue with whether mammograms prolong life is different than the issue of whether mammograms accurately predict cancer. Depending what research you look at, other approaches are sometimes seen as just as effective as mammograms. This is very controversial right now among people who do cancer treatment.

The point is, however, the article you cite does not say for every correct treatment there are 10 unneeded treatments; it suggests that for every life saved there may be 10 unneeded treatments. The summary doesn't say what they mean by a life saved. Mammograms find a certain number of cancers. Other diagnostic techniques also find a certain number of cancers. These sets overlap. It may be that the authors consider the number of lives saved by mammograms to be the delta between cancers detected/treated due to mammograms and due to other diagnostic techniques. This seems to me to be a valid issue, but it has nothing to do with how accurate mammograms are in detecting actual cancers, which seems to be 80%. (So, for example, what if other diagnostic techniques have a 70% chance of detecting cancer -- this might lead to the 10 unneeded treatments per one life saved statistic)

The issue I have with the article posted above is that it claims the probability of a woman having cancer after a positive mammogram is 7.8% when actual results show this to be 80%.

I did contact a statistics site that had a similar statement about AIDS tests and their response was, Baysian statistics are correct and they saw no problem with telling people that a positive AIDS test only indicated a very low chance of having AIDS.

Yeah, that's a good point that numbers used in a made-up example might have an effect on what people think about real-world diagnostic tests. The example is probably copied from another web page such as [1]. If you're sure they got it wrong, maybe write to them and get it fixed, or at least add a link to a page about the real tests.

[1] http://www.yudkowsky.net/rational/bayes

Reply on news.ycombinator.com