Hacker News Reader

HNHacker News

Top New Best Ask Show Jobs

Comment by valdiorn | Hacker News Reader

Full thread

valdiorn·

Sensitivity and specificity! That title is a bit misleading :)

I too could write a test that identifies autism 97% of the time:

return Random.Get(0, 100) < 97;

View on HN

What they did is take a control group that's the same size of the autism group. Then they did what's called leave-one-out cross-validation (which is exactly what it says, train on the entire set leaving one out, then use that one for validation and do so all ways). The measure they chose (Accuracy) is not without its flaws, but it would not be susceptible to the one you mention.

For instance, if you took 100 negative and 100 positive and then ran your test, you'd expect an accuracy of something a little more than 50%.

Under this methodology, how meaningful would having more data points be with respect to statistical meaningfulness? The study only appears to use 17 people in each group (for a total of 34 people).

Yeah I wonder what the false positive rate is. Didn't see any mention of it though.

I think it's just the headline that's misleading, the abstract says quite clearly: "Moreover, machine learning algorithms classified individuals as autistic or control with 97% accuracy from their fMRI neurocognitive markers."

The "or control" doesn't tell you exactly what the false positive rate is, but at least it sets the upper bound at 3%.

No reason to reverse engineer. The paper has all the information you need. One false negative in the cross validation. No false positives.

Reply on news.ycombinator.com