Hacker News Reader

Comment by rfatnabayeff | Hacker News Reader

The technique is reported to be 85% accurate. That is, there's 15% chance of false positive. According to wikipedia (http://en.wikipedia.org/wiki/Melanoma#Epidemiology), upper bound for melanoma incidence is 20 per 100000 inhabitants. Consider that 100000 people used the technique to screen themselves. There would be 15000 of them that would got the positive test result and only 20 of them who have melanoma. Thus, for a person having the positive result the probability that he indeed has the melanoma is LESS than 0.013%. That is , the probability of that the technique is reporting B.S. is over 99.8%.

Influenced by recent HN post: https://www.sciencenews.org/blog/context/doctors-flunk-quiz-...

View on HN

There's not enough information to know what "85% accuracy" means. Is that false positive, or false negatives, or a combination of both, or misreporting?

I agree that presenting risks as percentages instead of real numbers is misleading.

But they say this device is better than primary care doctors so I'm not sure what that says about primary care.

Gerd Gigerenzer wrote a book in 2002 about clinicians inability to understand percentages and screening tests.

http://www.amazon.co.uk/gp/aw/d/0140297863?pc_redir=13991465...

http://plus.maths.org/content/reckoning-risk

http://www.amazon.com/gp/aw/d/0140297863?pc_redir=1399339535...

"But they say this device is better than primary care doctors so I'm not sure what that says about primary care."

Only that primary care providers are not specialists.

But your point is important: is using this device as a gateway to dermatologist referrals a good thing.

I'm not sure that's how it works when it comes to accuracy of medical screening devices.

Say if there is a device that offers 99.9% accuracy but is used to screen a disease that occurs 1 in 100,000, then applying your logic would mean out of the 100 people who were identified positive, only 1 actually was accurate, thus makes the actual result of the device 99% false.

The 85% here is likely to refer to that out of the positive results, there is a 15% false-positive rate.

Medical screening device accuracies aren't calibrated to how rare the actual disease is.

It's not about rarity of the disease. The problem with those percentages is that they're commercial bragging. The 99.9% accuracy is indeed very poor accuracy in case where there is 99.999% chance that one doesn't have a disease at all. It's just measuring the length of a bacteria with carpenter's tape ruler - the tool is too poor.

A great statistical point--but assuming they're marketing this to (local/small town) doctors rather than laypeople, is that worse than they'd do by "eyeballing" it?

I could be wrong, but it's not the false positives you need to worry about - it's the false negatives.

15% false positive is no big deal. You can get confirmation tests, biopsies, second opinions, etc etc. But if someone scans a cancerous mole and is told "nope, all clear"? That's worrisome.

There was a Healthcare Triage video [0] that talked about this sort of analysis.

[0] https://www.youtube.com/watch?v=UF1T7KzRnrs&list=UUabaQPYxxK...

Since the device costs $500, I doubt many people will be screening themselves. Sounds more like a tool for professionals in less equipped areas.

Whatever the number of people, the device is still 99.8% B.S.-teller for $500.

The fatal flaw in your reasoning lies in your assumption that the device will be used to screen for melanoma in the entire population. In reality, it would be used only on patients who show other signs that would normally result in a referral to a specialist (i.e., the chance they have melanoma is much higher than 20/100000). The likelihood of an accurate diagnosis is 85% when this population is examined, and not the general population. As the article states, the likelihood of an accurate diagnosis (85%) is the same regardless of whether it's performed in the traditional manner by a specialist, or if it's performed using this device by a physician who would have otherwise referred the patient to a specialist.

TL;DR: 73.6% of all statistics are made up.

Reply on news.ycombinator.com