Disks lie. And the controllers that run them are partners in crime (2012)
dl.acm.org
dl.acm.org
To me it sounds like it's the manufacturers were the true partners in crime here.
I think higher prices for enterprise-grade disks are more about reliability than NCQ behaviour.
There are drives that can maintain their write rate 'forever', but those are substantially more expensive, for instance the Samsung 970 pro, see this graph from a Tom's Hardware article, it's a good graph because it very clearly shows how much cache the drives have and then it's game over:
https://cdn.mos.cms.futurecdn.net/V5reyvV4UAwHxEMQBP2XWS-970...
A review of that drive:
https://www.tomshardware.com/reviews/samsung-970-pro-ssd-rev...
Here is a barchart from some of the older Samsung SSD's https://www.diskusjon.no/uploads/monthly_06_2017/post-5591-0...
https://www.diskusjon.no/uploads/monthly_06_2017/post-5591-0...
As soon as you deplete the SLC cache you are down to ~50MB/s
We do the same thing for automobiles - which have diverse value-add feature sets, but for which we've pushed the average safety level consistently up due to their important and ubiquity.
The rest of the world buys reasonably sized vehicles well served by the automakers.
Not exclusively, no. But a regulatory regime considering only occupants' safety and not the safety of the other party in crashes is inherently biased towards larger vehicles. And various other regulations favor large SUVs/trucks over cars, from historical fuel efficiency rules to more recent depreciation schedules.
Of course that growth rate is still nothing compared to some pickup/off-road models. Every time I'm in NA I am shocked by the size of these behemoths. The crazy part is that more and more people are buying these things in my corner of Europe, preferably if they live in an old village with roads that are barely 2m wide per lane.
Some of these child decapitators have worse visibility and maneuverability than (EU) trucks. Ffs, if an 8-year old runs infront of your F250's grille they become invisible. This would be a valuable area of regulation imo, instead of mandatory speed limiters and breathalizers.
I want and need something that I can drive out into the pasture, up the hill, to the row of oaks at the back of my property, whilst towing a log splitter, so I can saw down two oak trees and turn them into a cord of firewood that I can then load into the bed and drive back to my home to stack on the firewood rack.
You worry about your life and let other people worry about theirs.
If your hypothetical 8 year old's parents were worried about his or hers, it wouldn't be an issue to begin with.
There is no practical reason that pickups need to have flat grilles made of solid metal and very little practical reason to have the hood be so tall in the first place. You could have your 3000kg vehicles with a torque-heavy V8 without unnecessarily endangering others.
ps: You might think differently after the 100th time you had to emergency brake and drive onto the grass because of a pickup that takes up 1.3 lanes going full speed around a blind corner.
Of course you should ideally be free to change the firmware of the hardware you own, but for that you would have to develop firmware with the desired features yourself.
General price segmentation, however, feels dirty.
Each HDD and/or SSD is its own computer, with its own CPU (controller) and RAM (cache). Each video card is its own computer with its own CPU (GPU) and RAM (video RAM). Every USB controller is its own computer with its own CPU and RAM. Each NIC is its own computer with its own CPU and RAM.
And so on and so forth.
Do disks lie? Fuck yeah, and so does everything else. Much like the human world, everyone (everything) lies so the system as a whole can (hopefully) keep running smoothly.
That's not how you make a smoothly running system, it's how you make one that insists everything is fine while it's all going to shit.
You could call it "adversarial compatibility", and we owe a lot of tech progress to it.
ATA/IDE was introduced in 1986 and a replacement for CHS wasn't introduced until 2002 when the 8 GB limit of the multiple of the fake CHS sizes became a problem.
Same with all other hardware indeed, like printers for instance.
1. The kind of drives you’re using if you’re running a distributed consensus in production is going to be enterprise where these kind of games are played a lot less
2. Establishing consensus is time consuming - by the time peers ack it, you will have flushed to disk. In a common 3 replica scenario, that means 1/3 nodes is taken care of.
3. The probability of 2 disks failing simultaneously in ways that cause a problem is pretty small.
But yeah, if your concensus has to be durably written and your flash controller is lying to you, your concensus is worthless (even reading what you wrote isn’t good enough because again - controller can lie to you and satisfy reads from the buffer too)
A disk AND an UPS is probably the minimal combination that you can guarantee that your data is actually persisted.
Which leads me to another question, should it really be the responsibility of each component to gracefully handle a random, instantaneous power loss?
Couldn't we have a small power bank on the motherboard keeping, say, 2 seconds of power and signaling emergency shutdown to components?
In a sense, that is very similar to sending a SIGTERM to a process to allow it to gracefully stop.
As far as I know this mechanism is not used for write caches.
Personally I think it should be the responsibility of the operating system. All filesystem operations on modern computers should be atomic, and either entirely succeed or entirely fail. And the OS should work with hard disk manufacturers to make sure their devices meet whatever expectation the OS makes of them.
Hilariously, the OS already uses transactions for its own filesystem metadata operations. But they don't share that capability with end user applications, which end up either being quietly broken in the event of power loss (most apps) or horribly complex to make up for the OS's shortfalls (most databases).
There are dozens of solutions in userland, from git's content-addressing to on-disk B+trees. There should just be a simple, effective, correct transactional filesystem API provided by the OS that applications can simply use. The best part of this proposal: a solution implemented by the OS directly would probably be faster than what most people would implement in userland anyway. Its ridiculous this doesn't already exist.
There's simply no structural reason for power failure (or application crashes) to be able to put hard disk data into an inconsistent, corrupt state.
Finally, it’s unlikely that implementing it in the OS would be faster because there’s a bunch of flash translation logic and garbage collection that has to happen - much faster if all of that happens on the other side of the PCIe bus with a dedicated coprocessor with a dedicated SPI directly to raw flash.
We could start with a nice implementation in the OS. Probably with a way for applications to opt in and out. And then optimize over time. Optimizations could include better integration with the way NVMe devices actually work, rather than the way they pretend to work to the rest of your computer.
Additionally, apps can do whatever they want in custom partitions if they need better storage perf (databases for example do this). Getting lower level than that is probably doable, but you run the risk of becoming firmware that is tied to particular drive controller design
Remember that computer storage technology is over 70 years old, many many things have been tried/implemented in that time.
Has this been tried? It seems like an obvious idea to me, but operating systems have evolved remarkably little in the last few decades. Especially the filesystem.
Well that seems a bit over the top. You would pay the price of full consistency for every write, while just one write every 3 years will fail. It would be insanely slow and wasteful.
Not to mention, that would basically disallow any kind of OS or hardware caching, which is immensely important for 99% of use cases.
> the OS should work with hard disk manufacturers to make sure their devices meet whatever expectation the OS makes of them
This is a bit of wishful thinking. We've seen already on numerous cases (Nvidia?) that OS developpers have very little leverage on hardware manufacturers. The best they could do is issue some kind of "Linux certified" label, which honestly nobody would care about.
Also, I guess there is some form of optimum that users and providers reach in terms of stability versus performance. If one OS provides x10 performance for 99.99% of uses while an other one provides 100% safety for /10 performance, I'm not sure users would choose the latter.
Why would it disallow caches? Just issue writes to a new location on disk and switch reads across to it after the write transaction has committed.
In any case, most writes aren’t performance sensitive. I’m thinking things like saving a file when code editing. My editor shouldn’t need to do some obscure, fragile renaming dance just to overwrite a file safely. It wouldn’t write 10x slower. Just have the OS do what it needs to do so my editor doesn’t have to.
And big transactions (like installing software or OS updates) could just do the whole update to a new location and then atomically swap to it. It shouldn’t cost any performance at all beyond a few pointers being moved around at the end of the transaction.
Maybe doing it for all writes is too much. Let databases opt out if they want to do their own thing. But I think it would be a much better default. Right now all good software does some version of this themselves. An OS level solution would allow all those hacks to be removed. If anything, I’d expect performance to improve.
You seem to be talking about _application level_ atomicity, like when you issue 4 write() to a file and want an "all or nothing" flush of these 4 writes.
I'm talking about OS level atomicity, as in "I want a write() (or fsync) to block until it is flushed to disk".
Am I understanding correctly?
> I'm talking about OS level atomicity, as in "I want a write() (or fsync) to block until it is flushed to disk".
Thats not atomicity. Thats a blocking API. And we already have an fsync that does that.
Blocking and atomicity are unrelated properties. You could have a blocking write thats not atomic (so if your computer turns off halfway through the write, it might have only half completed). And you can have an atomic write thats non-blocking (async). Or both or neither.
It's the same in this context. fsync uses a blocking API to implement atomicity. That is, after an fsync() all previous writes should have been written.
> we already have an fsync that does that.
Did you actually read the article...? The whole thing is about the fact that disk controllers can eagerly notify that a write was flushed to disk while in fact it is still in the disk write cache, and could have been either partially, fully or partly written, thus tricking fsync().
No, it doesn’t.
In that context fsync is a write barrier. It doesn’t make writes atomic. The difference is this: If you issue a handful of writes then your computer crashes, some, all or none of the writes may have completed. And some writes might be “skewed” - where part of the write succeeded and part did not. Atomicity means a set of actions happens as one indivisible unit, and it either all happens or all doesn’t happen. Think database transactions. Filesystem writes are not like that. They are not atomic operations[1].
Fsync does not change the behaviour of the write system call. Fsync only stalls your program until all writes have been written to disk, at least assuming your drive isn’t lying to you. Just like a memory barrier. If your computer crashes before fsync returns, all bets are off.
[1] With the exception of aligned writes under the device block size. Allegedly. I’d still like to see this actually documented somewhere.
It was recently deprecated because almost nobody uses it.
https://learn.microsoft.com/en-us/windows/win32/fileio/trans...
Non-atomic stuff is simpler on the implementation. Even if the behavior is worse for the user, it's easier to write, and you have fewer requirements to satisfy. So, not having a real pressure from the customers, it's just easier not to do this.
Very few apps need more than the ability to atomically swap a file.
And for the complicated multi write cases, a lot of software just use sqlite which deals with it.
No structural reason no, but a lot of other reasons. You can install your favourite atomic file operations library in your PL of choice and run some benchmarks to identify reason #1.
It's usually a RAID card cache Battery Backup Unit, which keeps the cache contents till the power would be resumed.
HDDs doesn't have anything like this and enterprise grade SSDs have a super capacitor to flush the cache.
Add to this SMR shenanigans.
> should it really be the responsibility of each component to gracefully handle a random, instantaneous power loss?
IMO, no. Sure, theoretically it sounds nice, but nowadays you can have literally gigabytes of data in transit and it's hard to make sure what even if you manage to write it what it would be logically consistent.
> Couldn't we have a small power bank on the motherboard keeping, say, 2 seconds of power and signaling emergency shutdown to components?
You are asking for:
a) a bank of supercaps - pricey, require proper isolation, have a lot of stored power => fails with sparks at best; or
b) LiPo rechargeable battery - pricey, require proper isolation, can pillow => fails with toxic smoke and fires.
And still this doesn't guarantee the data would be consistent.
BTW, there was a thing back in the day, UPS which could be installed in the case (usually 2x5.25"). While it sounds neat, it's the worst of all other variants:
small capacity - no place for a big battery
needs servicing - every 3 years replacement for battery
still can fail with sparks and fire
Just like the numerous ups and raid batteries I've had that reported they were fine until the moment they were actually needed.
Some server disks might have a backup power source, but that's certainly not a consumer hardware.