Comment by james412 | Hacker News Reader

Comment by james412 | Hacker News Reader

Parent Full thread

I think your wires may be a little crossed. This is unlikely to have ever been true for magnetic drives, but there is a quite significant grain of truth (no pun intended) to it for SSD.

Consumer SSDs may indeed return an early indication that a flush is complete, but whether they are lying depends heavily on your notion of persistence:

- the SSD may return done once the data is buffered in RAM

- or once it has reached a temporary log

- or once it has reached a 'permanent' location on flash

- or once it has recorded the mapping of data<->location to some separate directory

- or once it has updated some separate root pointer pointing to the latest version of the log/mapping structures in some separate special storage

I don't believe any SSD waits until the last step above before returning done. Separately, the SSD:

- may have no electrical power outage protection whatsoever

- may have supercapacitors fitted

- the supercap may guarantee full durability of every acknowledged flush

- or only guarantee atomicity up to some prior acknowledged flush

- or only guarantee that the drive has /some/ data on it in /some/ state on the next reboot

SSDs are insanely complex

It is interesting that to remain competitive in benchmarks, SSD vendors have to sell you an incredibly complicated software system alongside their flash chips soldered to a circuit board. They can't fix your filesystem, because you won't switch (and why pay them when your OS comes with one for free). They can't fix your database, because you won't switch (you already pay Amazon a million dollars a month for it). So they have to implement all the complicated parts of a database or filesystem, then try to understand the stream of events that real databases and real filesystems send it, and then restructure them to win the benchmark. All of this with no source code and probably no real test suite, and it all ends up being slower than a database that could just write to the flash chips directly.

I am also "excited" when I think about how the SSD shards blocks across flash chips and adds erasure codes, then people put the SSDs in RAID to shard blocks across SSDs and add erasure codes, then people put a filesystem on top of that that shards blocks across arrays and adds erasure codes, then put an application on top of that that writes to 3 replicas and writes erasure codes to 2 more. Now your "hello world" text file uses 8 gigabytes of storage, but at least you only lose it if there's one bug in the SSD controller, your RAID software, your filesystem, or your application. Or a tornado hits the SSD. Oh well.

HN often contains articles like "abstraction is bad", but even their contrived examples aren't this scary. I'm still pleasantly surprised when I log into my computer in the morning and still have a .bashrc.

I'd been hearing rumors (possibly spread by the mfgs to scare enterprises away from cheap IDE drives) about it during the late 90's and early 2000's.

Then bfitz proved in in 2005, and it turns out SCSI drives sometimes do it too: https://brad.livejournal.com/2116715.html

Oh no, this one came from I think the win98 era. I read about it in some (paper!) computer magazines, I had no or barely internet at the time. SDDs had not yet been invented.

Once one vendor decided to screw us, the others had no choice. It was trivial for benchmarks to work around it, too: Just send more data so the cache becomes irrelevant. The whole episode took a few months at most.

I dont remember if they flat-out ignored the flush, or started a flush but didn't wait for completion.

Reply on news.ycombinator.com