Hacker News Reader

Wouldn't more context mean that less people trip their algorithms, since then they'll be more accurate? Of course, it could be that if 10,000 trip their algorithms they won't do anything but if only 100 do they'll investigate each of them. I think that's still a different sort of thing, though.

View on HN

That's a fallacy of datamining. More data does not equate better algorithms, it equates more false positives.

As an extreme example, imagine using just random data. Any pattern you look for will occur sooner or later in pure random data. The rarity of the pattern is offset by the amount of data you generate. More data -> more patterns. That doesn't mean the incidence of the real thing increased at all.

This in electrical engineering at least is SNR (signal to noise ratio). Noise is filtered, however the intent is to recover the signal cleanly.

Anything which is self-detrimental is likely to be part of the signal, not the noise.

Reply on news.ycombinator.com