Comment by userbinator | Hacker News Reader

Comment by userbinator | Hacker News Reader

Parent Full thread

This is almost certainly going to be 3 or 4 bits per cell flash, which has exponentially worse retention and endurance than 1 (SLC) or 2 (MLC) bits per cell for only a multiplicative increase in density[1], and requires even more (fragile) algorithms for error correction/avoidance/longevity. I wouldn't be surprised if this "400GB" was actually 512GB (2^39 bytes, or "512 real gigabytes"[2]) of raw capacity, with 112GB of spare area. OK for short-term "transfer" storage, like the camera applications alluded to in the article, but definitely not for long-term archival or maybe even medium-term. Perhaps there should be a separate category for devices like this: "pseudononvolatile".[3]

To parallel an old saying: "big, fast, reliable - pick two."

[1] The terms "triple-level cell(TLC)/quad-level cell(QLC)" are entirely misleading, since they imply only a multiplicative increase in the actual number of distinct voltage levels in each cell. One wonders if they are deliberately downplaying the associated reliability issues by making it seem multiplicative instead of exponential. These should really be called triple-bit/quad-bit or eight-level/sixteen-level cells.

[2] Remember when flash disks had real binary capacities? I have a 64MB USB drive that really contains 131,072 user-accessible 512-byte sectors --- and it's still working, because it's SLC flash.

[3] I've worked with NAND flash over the years and noticed something interesting: you can very easily find datasheets for SLC and MLC (2-bit) flash, which give the endurance/retention. But there is very little on TLC (3-bit) flash --- they're almost all NDA'd leaks, which a few years ago were already rare, but seem to have mostly disappeared now --- and basically nothing on QLC/4-bit. Why the secrecy? It makes one wonder if there's something inconvenient about this high-density flash, that they don't want people to know...

> To parallel an old saying: "big, fast, reliable - pick two."

In this case, perhaps they decided to pick "big" twice.

As Bunny famously remarked: ”You are not storing your data — you are storing a probabilistic approximation of your data”.

I think this is a short-term issue just because error correction is so good. We have many error correction algorithms that approach the theoretical limit for efficiency.

Eventually error correction will take more bits than it saves and the number of levels per cell will stall.

I would say our SSD's are actually much more resistant to errors than hard drives were. Because the error correction in spinning disks is generally crap.

With flash, manufacturers have been forced to include extremely good ECC schemes because cells die all the time. Since errors are generally random and happen in large numbers, the life of the drive becomes predictable instead of the death cliff we used to see with spinning disks.

What does error correction algorithms have to do with whether the disk spins or is solid state?

It just happens that hard drives still have crap error correction. Probably because most of their failure modes are catastrophic anyways.

Part of it is also Sandforce. The flash to make SSD's existed a few years before they came on the scene but they were the catalyst. They created a sophisticated controller that could do very good error correction and wear leveling, allowing the first practical SSDs to be built.

> I think this is a short-term issue just because error correction is so good. We have many error correction algorithms that approach the theoretical limit for efficiency.

Doesn't this mean there isn't much room for improvement?

> To parallel an old saying: "big, fast, reliable - pick two."

Small, surely?

Edit: I'm dumb. Big storage. Duhhh!

By "big", I assume he means biggest storage, not smallest form factor.

By the way, unless there is some technology I don't know about, microSD cards have consistently been the highest density storage devices for many years running.

By the way, unless there is some technology I don't know about, microSD cards have consistently been the highest density storage devices for many years running.

That's because they're not much bigger than the raw NAND flash die itself, encapsulated in plastic: http://bunniestudios.com/blog/images/microsd_lineup.jpg

Of course, biological data storage (DNA, proteins, etc.) has the potential to be several orders of magnitude denser ( https://news.ycombinator.com/item?id=4396931 ), but the technology is still in development and highly error-prone.

Got it. Thanks!

Reply on news.ycombinator.com