Hacker News Reader

Comment by taspeotis | Hacker News Reader

> For Q1 2016 we are reporting on 61,590 operational hard drives ... [t]hese days we need to purchase drives in reasonably large quantities, 5,000 to 10,000 at a time.

To me, that's an astoundingly large number of hard drives. But I realise there are probably much bigger deployments. Does anybody happen to know just how many hard drives Amazon or Microsoft have for AWS/Azure?

View on HN

I imagine maybe 10x more than this. The numbers actually aren't too shocking if you think of the mirroring of data that's required for maximum uptime (not even archival), and the occasional disk failure.

1,000 TB (1PB) can be easily handled across ~150 (6-7TB)HDDs for one copy, but 300-450 HDDs would be required for additional mirroring.

Largest tape cartridges out there are between 6 and 8.5 TBs, and cost around $22 per TB. That's only $22,000 per PB, and this is for high throughput cartridges like LTO7 or StorageTek Titanium. LTO5 is much cheaper.

Considering that the largest tech companies and major organizations routinely cut POs for several $100ks and are dealing with 100s of PBs of data across disk, tape, DVDs etc, it isn't outside the realm of possibility to have 300,000+ individual disks and tapes floating out there :)

Anecdotally, Google's file system Colossus uses Reed-Solomon 1.5x replication. So those 150 drives might only turn out to be in the low 200's.

And I remember reading a tweet from a Google engineer that they would be paged if their free storage dropped below 5PB.

I thought waking up at 3am, shambling to my desk and connecting to the VPN was bad. Imagine having to drive down to the datacenter and rack 200 hard drives.

Sweet, I appreciate that info :) I imagine 5PB fills up quite quickly for Google too!

At that scale, it's just a function of (number of ethernet cables) x (avg size of ethernet cables), rather than disk space in their data center, I'd imagine!

> 1,000 TB (1PB) can be easily handled across ~150 (6-7TB)HDDs for one copy, but 300-450 HDDs would be required for additional mirroring.

Mirroring is not used at that scale

https://www.backblaze.com/blog/reed-solomon/

https://blogs.msdn.microsoft.com/windowsazurestorage/2012/06...

That's all good, it would just increase the number of disks required to be purchased and the amount of electricity/cooling/floorspace to maintain them. It would add to the disk count, but not really affect cost per TB all that much.

How do cloud storage vendors guarantee triple-mirroring and uptime then? Lots of 2TB drives? Lies? :)

Does it say triple mirroring or triple redundancy? Take a look at the article from Backblaze for an overview of the math. In their case it might be called quadruple redundancy. 20 shards hold 17 shards of data with triple parity. Events destroying hard drives containing information about your data could happen three times, and you still wouldn't lose anything.

> 1,000 TB (1PB) can be easily handled across ~150 (6-7TB)HDDs for one copy

Yea, our latest storage pod (https://www.backblaze.com/blog/open-source-data-storage-serv...) has 60 drives at about 8TB a piece so we're pushing 480TB. Two pods are about a Petabyte, if you go up to the 16TB Hard Drives some of the manufacturers are testing, you can hit pretty close to 1PB in an enclosure, and Dropbox is actually doing that already with their "Diskotech" boxes (HN link -> https://news.ycombinator.com/item?id=11282948) - so folks are already getting more and more dense :D

Yev from Backblaze here -> They have a lot more :D

And yet, your quantities aren't enough to attract the attention of the WDC direct sales department?

> And yet, your quantities aren't enough to attract the attention of the WDC direct sales department?

We've just started heading down the direct paths, but yea it seems like the minimum order number to work direct with the manufacturers is about 10,000 hard drives per order. We aren't quite at that capacity and we don't like to keep inventory since we try to run pretty lean. Also the price for hard drives tends to drop by a small percentage monthly, so for every month we have excess inventory we purchased earlier, that could potentially be money left on the table. So our orders to be smaller than their minimums. We're inching towards it though :)

That seems like a silly minimum for WDC to have, unless the discounting is phenomenal (50%+) compared to other vendors.

I'm asking myself, even if I'm a WDC rep that is selling hundreds of thousands of PC hard drives and having an excellent quarter/year, why would I turn away the business of a growing company?

Tape cartridges can be purchased in packs of 20 from any corporate vendor and no order is too small to attract a rep's attention. I've seen deals (25th to 75th percentile) from $10,000 to $200,000, and the min/max deal range is from $2,000 to $500,000

I imagine it has something to do with demand or their partnerships with their distributors. This is conjecture, but I'd think that in order to keep distribution deals/channels up they might have deals in place that stipulate any order over 10,000 goes direct and any order under goes to preferred distributors. No idea though, but that would make a smidge of sense.

>> how many hard drives Amazon or Microsoft have for AWS/Azure?

I'd like to know YouTube's totals with 500 hours of video being uploaded every minute...

Tough to say, but if this link is right: http://video.stackexchange.com/questions/8850/typical-size-o...

Then being conservative, we can say an hour of video is 1-2GB of data, so between 500 and 1,000 GB of data uploaded (just to YT) every minute.

Accounting for mirroring and compression, 525,949 minutes per year * 1 TB per minute = ~526 PB per year

Ironically this would be about 65,743 8TB HDDs, which is close to the number in this article :)

They also store several copies of each video (different codecs and resolutions).

I had heard 1PB/day somewhere...

Reply on news.ycombinator.com