SSD myths and legends: write endurance | Hacker News Reader

SSD myths and legends: write endurance

storagesearch.com

88 pointswslh76 comments

In the middle of reading this, I saw a 2TB SSD Advertisement then realized SSD just consumable product, it is just like tires which just change it every 2-3 years, for safety concern.

I think that's the right way to think about it. And we should have already been treating hard disks that way too.

For quite a while, I've been saying that too. We have the SMART data to know when they're wearing, so why don't we treat them like race car tires?

http://www.standalone-sysadmin.com/blog/2012/11/flash-drives...

Is this not already the mindset? I worked at a data storage company for a while; everyone knew hard drives fail, because we had so many disks we would have a couple failures just about every day. Everything was designed to be fault-tolerant so we could squeeze the most life out of every spindle, but in the absence of fault-tolerance I'm sure we would have had a spindle retiring schedule.

Are there many companies that do not acknowledge this basic fact of operation?

Not to normal consumers. Crap, I've been on the same set of spinning rust for going on seven years.

Knock on wood :-)

Thats the right way of thinking about it but its not really feasible for consumers.

"Oh hey, this Laptop has a SSD drive which is super fast. You will have to replace it every year though and if you dont have a backup strategy you might lose all you data. Did i mention they are quite expensive ?"

You will have to make quite the effort to burn trough a consumer SSD in machine with normal use for a year. 256 GB * 3000 gives a lot of write information.

Indeed, the standard SSD in a consumer-type usage is 7+ years of write.

In theory yes, in practice i have been using SSDs since 2009 and had them both failing (different brands and tech) after about 2 years of daily usage. Now i see i probably use them more than the average consumer, but i cant remember the last time i had a normal HDD fail in one of my machines.

I've seen a number of normal HDD failures. I think that the first one that I've seen, been with a double-5 inch slot hard drive, which was failing to start spinning at times. You've had to turn the PC box abruptly, to make it spin ;)

But as of now, have to see an SSD fail. Longest running that I have now is a Linux box that've been up since Oct/2007. (Heh, cool part about that linux box - there are no moving parts at all - no coolers, no nothing. Even in the power supply ;) )

Sounds cool :) I am not saying i have never seen HDDs fail, but in the almost 20 years i have been computing heavily, it might have been about 3 in my own machines, while i went through 2 SSDs in 4 years. Id still never go back to HDDs as my system drive :)

I've had no hard drive failures in over a decade, but my first SSD (a Crucial M4) failed in about a year. I'm back to reliable hard drives.

Spinning disks share most of that; they're just cheaper per gigabyte and slower. A spinning disk (is not and has never been sufficient to store important data.

At least with SSDs the form factor can be a lot smaller, so we might see slower, cheaper ones appear more often as external drives for true portable backups. Having your data on two SSDs is better than one spinning disk, especially if one of the SSDs is usually unplugged.

I just spent quite some time assisting someone with a 1TB WD drive that was failing after 1 month. No backup strategy, quite expensive.

Can you PM me what brand it is?

'WD' means Western Digital.

Hacker News doesn't have PMs. Leave contact info in your profile if you want to be contacted.

I've had my Intel SSD since 2009, it's only needed to be replaced once, and Intel did it in a week (and I didn't lose any data). SMART warned me that it might fail soon (I think it ran out of remappable sectors), and Intel sent me a free replacement, which has been running fine since.

I got an Intel X25 in early 2010 and used it as my daily dev machine while averaging probably 60 hours a week for tasks like cloning large DBs and parsing gigabytes of log files, and it's still running strong.

Yep, that's what I have too. X25-M, 160 GB. I remember I paid 400 GBP or something for it back then.

Me too, had the 80GB model which failed after 2 years in 2011, got a Crucial M4 since then which starts to get some occasional freezes and write errors.

I also got 2 Supertalent MLC Drives in 2009 for my brother and a friend, both died after ~2 years as well. Now we are all running the Crucial M4 which are fine, for now ;) I love SSDs, have been an evangelist since 2009 but i dont trust their reliability :)

For those in the know, and for the benefit of laymen who don't have the time to do the research, what are the things that a buyer should look for in a SSD, in regards to reliability and longevity? Technology, brand, controller?

the only critical thing is physical size. some (mainly ultralight) laptops take drives that are thinner than normal laptop drives (this is also an issue if you're buying a new spinning disk drive, of course).

apart from that, assuming you buy a popular, current model, it will just work (and be reliable). this article is bikeshedding for hardware and enterprise geeks. so, just like anything else, get something reasonably priced from a reputable make (longevity of support, availability). currently samsung seem pretty good.

and finally, if you don't already use one, get one. it's by far the best way to improve "perceived speed" of your computer (well, assuming you already have enough memory).

My Intel X?? came as a superslim 2.5 package, but with a plastic bumper as part of the construction making it up to full height. Having to disassemble it first thing to remove the buffer went just fine, but wasn't a great start (warranty voided before I even powered it up?)

Had to remove enclosure of a Kingston drive to fit a notebook also. Just cycled all the storage in the desktop before voiding my warranty... Still sucks

In your case it's double ironic since Intel is the one that pushed for slim ssd because of the ultrabook shenanigans while everyone was doing a standard size. Not that having smaller drivers is bad...

The only thing you have to know about reliability is "always have backups". I bought 5 SSDs and used them as OS drives on laptops and desktops. No SSD actually failed, but they had "glitches" - 3 of them, of different brands and on different PCs and OSes, lost data for no apparent reason - nothing in the SMART logs and tests. Twice the OS had to be restored from backups. The good news is that after the restore (on the same device) everything was fine and I continued using them.

Write endurance is very real.

http://www.ssdaddict.com/ss/Endurance_cr_20130122.png

http://www.xtremesystems.org/forums/showthread.php?271063-SS...

The data given on the second link does not inspire confidence in consumer SSDs

How so? The tests are in no way normal usage, most writing hundreds of terabytes or more before failure, exactly in the expected range for those sizes of SSDs.

Consumer devices do not write remotely as much as people seem to think they do. Heck, I ran a very large, endless-churn financial system on a FusionIO platform, and two years in it had barely written several hundred terabytes.

I was looking at SSDs as scratch disks and the like, but some of the examples failed in months. Its an electronic device, sorry but other than purposely putting in an environment that is not conducive to its long term health, like heat or dunking it, I should be able to read and write to it to my heart's content.

I fully understand commercial grade equipment versus consumer, but in the world of no moving parts it begs the question, what does it take to make them more reliable. Because some of the devices didn't last long enough for me to lose the receipt

On the first link, what's the red bar represent?

If anyone's interested in some more details for write endurance I'd recommend Anantechs SSD reviews. http://www.anandtech.com/show/7173/samsung-ssd-840-evo-revie... .

If the numbers in those are accurate it looks like write endurance shouldn't be a problem at least for any consumer drive. The estimate on that review was 12 years in a typical consumer setup, and I'd reckon that almost everyone would have replaced a laptop or PC in that kind of timeframe.

Can someone explain the discrepancy between

> I still hear stories of users thinking it's perfectly normal and economic to replace burned out Intel SSDs every 6 to 12 months

and

> How long have you got before the disk is trashed? ... 51 years!

Ditto.

Also, assuming it's 51 years, that generally means I have nothing to worry about with an iPhone or (new) MacBook, right?

Assume that any hard drive can go away at any time, for any number of reasons. Especially portable devices like those. Always use some sort of automated backup to a remote location for data you want to keep around.

Because time correlates only very loosely with SSD failure, it's all about writes and people write at widely varying rates.

I thought the 51 years was supposed to be worst-case.

There's a big difference between using consumer grade MLC flash and running a very high intensity workload and using enterprise grade SLC flash and only using 80 MB/s of bandwidth.

As a consumer you really don't have to worry about write endurance, though. I mean, how much do you write to your drive every day, maybe 5 GB at most? Even with MLC flash and a fairly small 120 GB drive that'll last you for a couple of hundred years. And even when it fails you can still read the data out, you just can't write new data. The problem you should worry about is a firmware bug in the part of the drive that handles the wear leveling abstraction. I haven't seen one of those in a while, but when many companies were just starting to develop SSDs they occurred fairly frequently, though usually the manufacturer would figure out the problem and patch the firmware for new drives after enough failures in the field.

But, is a hybrid approach the best option? I mean an SSD for the software and an HD for the data?

I actually think the best approach, longterm, would be to use the SSD as a cache for the HD at a filesystem level. I believe something along those lines was just merged into the Linux kernel.

Yep, dm-cache essentially does what Intel Smart Response does except it's done in the OS and not the BIOS.

If the SSD part fails you pretty much lost all benefit of the HDD itself. It might or might not function with the bad flash chips but you definitely lost all the performance boost. The caching options sound much better overall but they do take more space (2 slots vs. 1 for the hybrid).

With hybrid I didn't mean those hybrid HDs but two separate storage devices using two slots.

Just a quick note: before you buy a "self-encrypting drive" (SED SSD), do your research and make sure you get one where you can control the keys.

A couple of months ago, I spent almost $400 on a Crucial M500 480GB SSD to put in a new laptop only to discover that apparently the drive itself handles the generation of keys.

Thus, technically all of the data on the drive is encrypted but you can take the SSD out, put it into another machine, and access the data without any issues. Total marketing ploy.

The point behind it is you can delete the key in one place and not have to worry about erasing every part of flash in the machine (which, after remapping, may not even be possible through normal means).

What are these RAM SSDs of which the article speaks? I searched around for a bit, but couldn't find any place to order one (keep getting hits for PCs for sale)... are they available in say, 64GB in SATA?

They tend to be enterprisey or OEM-only; here are some examples: http://www.ddrdrive.com/ http://www.stec-inc.com/products/zeusram-enterprise-sas-ssd/ http://www.dataonstorage.com/dataon-products/enterprise-clas... http://www.sandirect.com/texas-memory-ramsan-440-enterprise-...

8GB for $3000. And I thought SSDs were expensive ...

But they are typically a factor of 5-10 times faster than current generation SSDs, and not just in raw read/write, but also in access time.

That's more expensive than just running on regular memory.

Because it's nonvolatile.

Obviously if you're willing to give up the reboot/etc. tolerance, you can run a ramfs and buy more ram, especially on servers or desktops (it isn't too hard to get absurd things like 18 DIMM slots in a machine)

He's referring to Ram disks in the article. (http://en.wikipedia.org/wiki/RAM_disk).

The only result I found to buy was this software from AMD (to partition some of your RAM as a RAM disk). http://www.newegg.com/Product/Product.aspx?Item=N82E16832385....

ImDisk[1] is free without restrictions and also open source. I just set it as a small system startup script[2] to create a ramdrive on boot into Windows. Makes for a nice throwaway drive for things that are read/write intensive (like compression/extraction) or putting a mini virtual machine on and running it purely in RAM. I also put things like my temp downloads there, page file, temp directories, browser caches/history. I suppose you could put the scratch drive for Photoshop on it as well, but I have never tried that.

Technically, you can also save all the files from the disk on shutdown, but you have to wait for them to copy back, though I prefer the volatile nature of the drive and having it wipe everything on reboot except for a predefined directory structure that gets recreated each boot.

RAM is cheap and I ended up buying 24GB for $150 or so a while back so having a ramdrive gives you a use for all that excess RAM. When I'm using my laptop with 4GB I miss having the extra space a bit.

[1] http://www.ltr-data.se/opencode.html/

[2] https://gist.github.com/yareally/6199239 (my batch startup file example and link to a more detailed guide)

You put your page file in a Ramdisk? We need to go deeper...

Mainly to avoid the Windows page file from getting fragmented on a HDD, since fragmentation on a RAM disk is a non-issue. I keep my home desktop PC on most of the time and don't reboot often unless switching OSs.

If you have RAM free, Windows shouldn't be using your page file. If you have so little RAM free that Windows is paging, then using even more RAM to host the pagefile will make things worse. You might as well just turn off the pagefile entirely.

Windows likes to use it regardless of the amount of RAM you have free. Page File on Windows is unfortunately not as friendly as the swap area on Linux (where what you said is true).

Currently I have 8 GB free (out of 24 GB) and the page file is sitting at 944 MB (up from 96 MB) after rebooting about 12 hours ago. Windows has always been this way regardless of the version.

Turning it off is also not a good idea as Windows does use it regardless of free RAM.

If moving some data to disk helps performance by freeing up RAM, then putting the swap file back in RAM won't help because it's just taking up RAM again. Or, if you're just worried that Windows is putting things in the swap file that would perform better if they were kept in memory, it would be just as effective to disable the page file to force them to be in RAM. Either way, there's no improvement.

If you read my previous posts, the page file is far from the only reason I have a ram drive. Since I use it for lots of other things and I rarely use all of it, I might as well put my page file there as well. I also have 24 GB of RAM so not overly worried about storage.

I regret mentioning the page file, because it always starts some sort of silly debate on any forum it's discussed (Google yields tons of results).

Windows does things like swap out applications which are minimised or not recently used to allow more space for disk cache. You're neatly defeating the point of swapping them out by keeping the page file in ram.

You are still shuffling pages in ram in order to "swap" them though. The gp is correct. To achieve your goal, disable the page file. Having it on the ram drive entirely defeats its purpose; The ram backed page file won't help you in the case of memory exhaustion, and it doesn't allow the OS to put the ram to better use where it judges that possible. It's your machine and your perogative, but it is daft, and people are going to point that out.

Defeating it is a good thing. I don't want to waste time waiting for programs when I unminimize.

And windows gets cranky without any swap at all. When I 'disable' swap I always set it to 20MB or something.

> If you have RAM free, Windows shouldn't be using your page file

http://serverfault.com/questions/23621/any-benefit-or-detrim...

    Many people seem to assume that Windows pushes data into the pagefile on
    demand. EG: something wants a lot of memory, and there is not enough RAM to
    fill the need, so Windows begins madly writing data from RAM to disk at this
    last minute, so that it can free up RAM for the new demands.

    This is incorrect. There's more going on under the hood. Generally speaking,
    Windows maintains a backing store, meaning that it wants to see everything
    that's in memory also on the disk somewhere. Now, when something comes along
    and demands a lot of memory, Windows can clear RAM very quickly, because
    that data is already on disk, ready to be paged back into RAM if it is
    called for. So it can be said that much of what's in pagefile is also in
    RAM; the data was preemptively placed in pagefile to speed up new memory
    allocation demands.

And this is another good reason to have at least a token pagefile. Without any virtual memory all the memory a program requests gets allocated right away.

    Removing pagefile entirely can cause more disk thrashing. Imagine a simple
    scenario where some app launches and demands 80% of existing RAM. This would
    force current executable code out of RAM - possibly even OS code.

There are references for more detailed reading listed in the provided link.

edit: I also do the same thing as yareally with my pagefile, though I have much less RAM than him.

And if you use a 32 bit windows but have more than 4GB ram, you even can use the otherwise unusable RAM for the RAM disk.

I don't think he's referring to RAM disk software drivers as RAM SSD's because he says:

> where as long as enterprise users remember to replace their batteries periodically...

There was a piece of consumer hardware that I used for a very short time called the iRam. You basically got a battery backed RAM based disk (presented over SATA) that could saturate just about any application you wanted. Was expensive as hell and only went up to 4GB, but I would love to have something similar today.

http://en.wikipedia.org/wiki/I-RAM

How does this affect cloud providers using SSDs? Are they using the reliable and expensive version? Do they use consumer ones and then swap them periodically?

How does everyone else set up their dev systems? I just rebuilt my dev machine and I'm using one SSD drive for the OS, one SDD for data (database, Cassandra stuff), and then a regular drive to hold code and builds since that gets rewritten several hundred times a day sometimes.

One SSD of Vertex 4 speed or higher, usually in a laptop, max RAM. Haven't seen the need for a workstation in over five years. Have gone through 4 laptops for processor increases/lightness/battery in that time, so SSD is always replaced long before anything could go bad anyway. Develop Android/iOS mainly, although there's often a Jetty/Rails/MongoDB/NodeJS/Postgres server or two and a VM running.

> I just rebuilt my dev machine and I'm using one SSD drive for the OS, one SDD for data (database, Cassandra stuff), and then a regular drive to hold code and builds ...

So instead of one large storage pool, you have three. This means your chance to have your system grind to a halt with a disk-full condition is three times greater than it would otherwise be.

I've never understood this logic, but I certainly see it implemented a lot. It's a new version of disk hard-partitioning in *nix systems -- a scheme that stopped making sense once drives because large enough to contain all the resources on one unit.

With one drive containing all stored resources and no hard partitions, there's exactly one disk-full failure mode -- when the disk is full.

When Charles Lindbergh was asked why he wanted only one engine on his historic Atlantic-crossing plane, he replied that, since the multi-engine planes of his day couldn't stay airborne on one engine, his design was actually safer.

If he combined them, it would be likely he would have to move to an HDD to fit everything. The part that's really unfortuante about that is that HDDs do not have the best reading/writing since the heads actually need to seek the data. These creates performance bottlenecks, especially if he's trying to transfer a build while the database is active.

Fewer disks, less chance of failure. it's extra important to back up though, since one disk failure can result in entire catastrophe! Losing an OS but still having DB and Code isn't that catastrophic.

I used to have the same opinion you did until I started doing some really filesystem intense stuff (not even big, just lots of creating and deleting of files) and suddenly my entire system ground to a halt. Huge delays (top showed state "D"—waiting for device).

I moved that stuff to its own disk and my system became smooth as butter again.

I'm still not down with the traditional unix '"/usr" must be its own partition!' thing, but it does help to separate your tasks onto separate disks.

I use Intel Smart Response SSD caching on my work PC - 1 x 500GB HD and 1 x 60GB SSD as cache. (I think you can set this up to cache multiple drives but you might need one SSD partition per HD.) It was a little fiddly to set up but since then that it's been trouble-free. It was not expensive to buy the parts and it does a good job of balancing the competing requirements of disk space, performance, cost and general hassle.

Flat-out performance is a little ropey by SSD standards, but random accesses are fast, so it mostly feels SSD-like. (I have the system set to write-back mode, so write speed is decent too.) Of course, as with any cache, it is possible for it to become filled with inappropriate data, or for a data set to exceed the size of the cache, but for my purposes (mainly building C++ and running the result) I've found it to do a good job. With a 60GB cache I've found these situations pretty rare day to day anyway.

(Smart Response is Windows-only; I don't know how it compares to the Mac OS X or Linux equivalents.)

Zsolt Kerekes is a pretty sweet name. I don't think there is a name that says "solid state storage" better than "Zsolt Kerekes".

It is a simple Hungarian name and translates very roughly to Solt Wheeler.

http://en.wikipedia.org/wiki/Zsolt

I meant that as a compliment! I literally think it's a pretty sweet name. Sorry!

Reply on news.ycombinator.com