Comment by Veratyr | Hacker News Reader

Comment by Veratyr | Hacker News Reader

Parent Full thread

Something important that I don't think many people bring up is the cost of these drives in a RAID, taking reliability into account.

When you're using this as the sole drive in a desktop machine or something like that, paying the extra for a HGST is pretty straightforward, but I see too many people passing over Seagate or WD when they intend to put them in a RAID, for "reliability"'s sake, without (it seems to me) much thought into whether it's really worth it.

Sorting by $/GB for Hitachi, Seagate and WD on drives >= 4TB (http://pcpartpicker.com/products/internal-hard-drive/#m=19,3...), you get:

- 4TB WD: $129

- 4TB Seagate: $114

- 4TB Hitachi (HGST): $158

So in order to get the reliability of a HGST, you're paying a 38% premium, is the extra reliability helpful in a RAID?

If you look at say a RAID6 array, aiming for usable storage of say 16TB, HGST's longest running drive (at 0.11% failure rate) would have a roughly 0.000016% chance of failing in a given year (0.0011^3 * 6 * 5 * 4) (chance of first drive failing = 6x(single), second=5x(single) etc.), assuming nobody replaced a failed disk, since you need 3 drives to fail.

Now an array of Seagate's longest running drive (2.66% failure rate) of the same size would have a chance of failure of ~0.225853% in a given year.

But, the cost of the disks in the Hitachi array would be 6x158 = $948 while the price of the disks in the Seagate array would be 6x114 = $684. For the price of the Hitachi disks you can get 8 Seagate disks.

So what happens if you just add the extra Seagate disks to the array as extra parity? Now you need 5 drive failures, giving an equation that looks like ((8 * 7 * 6 * 5 * 4 * (0.0266^5)) * 100), and a chance of failure of 0.008949%, still way higher than the Hitachi.

In the end buying Hitachi/HGST seems like the right choice anyway but I thought it interesting since I hadn't seen anyone else look at things this way.

If anyone has any problems with my math, please feel free to point it out, my stats background is pretty limited.

I think a better way to look at it is not the chance of total data loss, but the expected running cost over time. Even with just RAID6 it's hard to imagine a scenario in which you would be unable to replace one of the failed drives and resilver before getting two further failures, so the naive reliability calculations are extremely biased.

So if we assume that we can safely operate at some fixed level of redundancy (say 3 copies) across all hard drive vendors, the only question is (how many drives you need to replace per year) × (the price of those drives).

The problem for me at least is that most of the drives I use come with 3ish year warranties, so they're RMA'ed for free. By the time I'd have to buy a replacement I'd be intending to upgrade anyway.

RAID6 is unusable for many types of "always on" data, in particular databases backing high volume sites; the performance drops so much during a rebuild that for many uses it might as well be offline.

Then you use mirrors, you can apply a similar calculation there. Lower price per disk means you can put more disks in each mirror and compensate for lower per-disk reliability.

Obviously the type of redundancy used will depend on your specific application requirements, including performance.

Good analysis and I've wondered this myself, though this assumes failure rate is linear rather than accelerating over time, that failures are independent (vs a design flaw that causes them to fail around the same time, which happens enough that people recommend buying drives from separate batches / date of mfg), and you have to factor in the cost of a larger RAID array + power to support the same level of reliability. It may make sense in data centers but the smaller you go, the bigger the jump in cost from drive arrays (e.g. a synology 2-bay, 4-bay, 5-bay, and 8-bay are each pretty significant jumps). I too am on team HGST because of that.

>though this assumes failure rate is linear rather than accelerating over time, that failures are independent (vs a design flaw that causes them to fail around the same time, which happens enough that people recommend buying drives from separate batches / date of mfg)

I saw this happen once. A drive died, it was replaced and while the array was rebuilding several more failed. They ended up losing the entire SAN.

It wasn't a good day.

This is why I try not to buy many disks at once, just buy one at a time, and if I need more than one I'll go to different stores. Too much chance of getting a bad batch of disks and having the whole array go down (I've had that happen too).

Of course, that doesn't work at a large scale, but then you have other methods of increasing redundancy.

Yeah it's by no means perfect. Another problem that comes up is that when one drive fails the load on the others increases, which can lead to more failures. It'd be really interesting if someone's looked into the stats here but I haven't seen it.

As for paying for the power+drive bays, that's true of the consumer NAS devices but if you buy used servers instead it isn't such a problem. For the price of a 12 bay Synology disk station you can buy a 24 bay, 4U, used Supermciro 846E16-R1200B with a couple of CPUs and 144GB of RAM.

Not a great choice if you're short on space but can be decent otherwise.

Reply on news.ycombinator.com