Hacker News Reader

With a range that large, I would be highly suspect that when you did get 90%, it was just fitting the data. And how would you even confirm if they had the flu - even a doctor could misdiagnose mild flu, which adds more noise.

60% really means very little if it's a binary choice - flu or no. You'll be right 50% of the times no mater what.

View on HN

>60% really means very little if it's a binary choice - flu or no. You'll be right 50% of the times no mater what.

What? Just because something is binary doesn't mean it happens half the time.

Hmm, yeah... good point on that. I'll leave my comment, but I take it back :)

With severely unbalanced classes, overall accuracy is not a good metric. So we use recall of the symptom class, and how it varies with the precision. Recall = 0.6 to 0.9. All results are based on cross-validation.

There is a lot more sophistication we can add (e.g. markov properties), these are almost first-order results.

Nonetheless important to validate with other populations. That would be a big part of testing any product.

Reply on news.ycombinator.com