Hacker News Reader

Thresholds are often useful with these kinds of stats. Aka a drive moving one sector might mean nothing, but moving 30 in a week could be great predictor. Further they only have 70k drives across a range of product lines so what predicts drive X failing very well might say little about drive Y.

PS: Rememebr all RAM gets bit flit errors over time. Which is one of the reasons rebooting is often so useful, but also means one off errors are often meaningless.

View on HN

ECC ram will either correct the error or can raise a non maskable interrupt if it can't be corrected.

ECC vastly lowers, but does not remove this problem. You can for example get bit flip errors while doing the checksum.

As a personal user it's a non issue but scale things to ~70k devices and you get ~1.7 million device hours per day.

Fair enough. Your original comment did not make this distinction.

I dug and found https://www.fiala.me/pubs/papers/sc12-redmpi.pdf the title of which is "Detection and Correction of Silent Data Corruption for Large-Scale High-Performance Computing."

They state that for a cray there was a double bit flip about 1x/day for 75k modules. To (probably incorrectly) extrapolate, if you have a server with 16 modules that would be equivalent to a single double failure about once every 13 years.

Reply on news.ycombinator.com