If I were their customer, I would be looking for alternatives right about now...
They achieve the best bang for buck by using ordinary consumer hard drives in RAID arrays. During the hard drive crisis, the cheapest way of procuring these was by purchasing external drives from ordinary stores.
There's nothing wrong with that. If anything, it shows agility and ingenuity.
What else can Backblaze do? Stockpile drives? The hard drive industry moves fast, with major product releases every 6-12 months and constant price reductions; this would almost certainly increase the cost to the end user.
No business can adequately prepare for a supply shock like this. Imagine airlines facing a 300% increase in the price of fuel. Almost every single one would go out of business.
In this case, would you be berating the ingenious airline that goes out of its way to track down fuel at a cheaper price, finding a low-cost fuel that they can easily refine themselves to aviation fuel, so as to honour their commitments to their customers and avoiding raising their prices?
No. They use hedges to control their cost (this is the original point of hedging...). If a company is reliant on a constant supply of hard-drives and doesn't want to be exposed to the risk of extreme price jumps it should probably do the same.
e.g. http://www.delta.com/business_programs_services/delta_cargo/... the first column is the fuel surcharge that floats with fuel prices.
The Backblaze article clearly said that paying more for drives and passing it on the customers was an option. They just found a better one. They didn't mention it, but just losing money for a while is also a reasonable option if you don't want to antagonize your customers.
Given that hard drive prices almost always drop constantly, is that tactic really a good idea? Is it possible to reasonably hedge against price spikes in a commodity whose price you expect to consistently and rapidly drop? (Serious questions, really.)
Typically if you're hedging a commodity for actual use, what you're trying to do is to lock in your cost and not to make money out of it. Let's look at an airline for example. The price of oil currently is $90/barrel and I think it might go up even more to $120/barrel. I buy a contract for 100 barrels of oil at $100/barrel deliverable in 3 months. Say I use 100 barrels of oil every 3 months. I now know that my fuel cost is now $10,000 for those 3 months. This then allows me to price my product based on that $10,000 assumption.
In the case of hard drives, I have no idea if it would be a good idea to hedge since the prices keep on dropping and the Thailand flood is not an everyday occurance and in general prices of hard drives are pretty stable and don't fluctuate like oil prices. But fundamentally, you're not trying to avoid losing money; you're trying to lock in your cost so you can price your service accordingly and not be in a bind if the price of your commodity fluctuates.
In this case internal drives are ordinary aviation fuel, external drives are the alternative fuel, the price of which has been largely unaffected by the supply shock, and the process of taking the external drives apart to reveal internal drives is the equivalent of a simple refining process that turns the alternative fuel into ordinary aviation fuel.
In the end, they are cheap, if you are worried about their methods I'm sure they have competitors who approach things in a more traditional fashion and charge accordingly.
They have built a backup architecture designed to use low reliability commodity hard drives.
This post shows the lengths they go to just to keep growing. Imagine what they'll do to keep the system running!
I suspect, though, without any actual data to back this up, that the drives that go into (low-cost) external HDD enclosures have lower MTBF ratings than the desktop-oriented internal drives. I think that the usage pattern for an external HDD is very different in terms of longevity in operation than desktop hard drives (and these have lower guarantees than eg. server-grade SATA drives).
If I was a manufacturer of such enclosures I'd be willing to buy cheaper drives with lower promised MTBF for embedding them in enclosures that stay off most of the time.
So, even if they're the same, I guess the manufacturer performs some tests and sells drives that fail some non-critical ones at a lower price (pretty much what Intel and AMD do with their chips - those that can't sustain the highest frequencies are downgraded to clock rates that are stable).
edit: The comment from @dsr_ tends to support my hypothesis, although I haven't looked at the differences in guarantees of external enclosures, it's true that internal desktop HDDs can have up to 5 years of guarantee (i.e. WD's Black series has a 5 years guarantee).
It's why their restore isn't instantaneous. You have to ask for the files and they'll let you know when they're ready, this takes time (up to several hours).
AIUI they essentially fill up each Pod with incoming data and, when it's full it goes into a low power mode where they spin down all of the disks in that Pod.
Restore requests (from many users) will usually span quite a few Pods so they batch up requests for each Pod so that can minimise the number of times that drives in a Pod need to be spun up to copy off files. Rather than spinning up and down once every 10 minutes to service 36 requests over 6 hours they wait 6 hours, spin up once and service all 36 requests for that pod in one go. (Figures are for example only.)
(Small restores may be instantaneous if the Pod is still being filled or just happens to be spun up for restore soon. Large restores will more than likely span multiple pods and so there's more of a delay until all parts of the restore have been obtained from all of the pods involved.)
If anything this maps pretty well to the expected usage pattern of an HDD in an external enclosure as many of them are used for infrequent backups rather than extending constant use storage. It's how I expect Backblaze will be hoping to get much more than the usual MTBF out of their drives they have purchased.
HDD guarantees may be time based, but HDD failure rates are linked much more closely with usage patterns; number of spin-ups/downs, number of seeks, number of sector writes, etc. A drive in constant use on a desktop machine is going to be hammered a lot more than one that's essentially written to just once (as the pod fills up) and then spun up every now and again for partial reads.
(I know nothing about using tapes to seriously store data, so this is conjecture.)
Experience: I helped route and process data for the Large Hadron's CMS detector.
The machine plans are on their site and they explain why they chose off the shelf components. My only regret is that no one on our team has welding skills, so we couldn't make awesome face plates. lol
Backblaze is one of the more transparent companies operating in the space, which increased the trust level.