Comment by Strilanc | Hacker News Reader

Comment by Strilanc | Hacker News Reader

Parent Full thread

To rant a bit more about this one second delay thing.

This reeks of a type of thinking where you are relying on other parts of the system to compensate. You might expect to hear things like "it's okay, the safety driver will catch it". Speaking for myself personally, this type of thinking comes very naturally. I like to come up with little proofs that a problem is handled by one part of a program, and then forget about that problem. But in my experience (which does not involve writing anything safety critical) this strategy kinda sucks at getting things right. Dependencies change, assumptions become invalid, holes in your intuitive proof become apparent, etc, etc, etc, and the thing falls over.

If you are designing a safety critical system, something you really want to work, I don't think you should be thinking in the mode where each problem is handled by one part of the system. You need to be thinking in terms of defense in depth. When something goes wrong, many parts of the system should all be able to detect and correct the problem. And then when something bad does come up, and 9 out of 10 of those defensive layers each individually were sufficient to save the day so there was no disaster, you should go figure out what the hell went wrong in the tenth.

> Dependencies change, assumptions become invalid, holes in your intuitive proof become apparent, etc, etc, etc, and the thing falls over.

I apply encryption to storage. I can't tell you how often people try to push back on encrypting storage with stories like, "But we have access controls and auditing in place. And when we have a deprovisioning process for our drives. Encryption is costly and redundant, so why should we do it?"

Through the years I can recount several after-the-fact incidents where encryption ended up saving their bacon because of weird and entirely unanticipated events. One notable one was where a hypervisor bug caused memory to persist to an unintended location during suspend/resume, and the only reason customer data wasn't exposed in production was because the storage target was encrypted. In another case the "streams were crossed" when assigning virtual disks to virtual machines. The (older) base disk images weren't encrypted in that case, but because the newer machines were applying encryption in the backend before the blocks were exposed to the guest OS, the "unencrypted" disk content came across as garbage (plaintext was "decrypted," which with the algorithms we were using was equivalent to encrypting), again preserving the confidentiality of the original disk images.

The concept of "belt and suspenders" is often lost on people when it comes to safety and security systems.

It's depressing how much trouble some people have in understanding the idea of defense in depth (https://en.wikipedia.org/wiki/Defense_in_depth_(computing)).

Oh, you have access controls in place? Great. What happens if they fail?

Oh, you have a deprovisioning process in place? Great. What happens when someone doesn't follow it?

Systems fail all the time. If your defense only has one layer, when that layer fails (and it will, eventually) you're SOL. Multiple layers of defense give you resiliency.

> This reeks of a type of thinking where you are relying on other parts of the system to compensate.

This is what Boeing did with Max. The airframe wasn’t stable in and of itself, and they relied on software to compensate. Terrible idea.

>And then when something bad does come up, and 9 out of 10 of those defensive layers

And there should be a definitive priority established between those layers so that, if one fails, the other 9 don't attempt to correct in different ways. It should fail from the most conservative to the least so that a false positive results in erring towards stopping the vehicle.

This. The right way to think is that each component, in parallel, have a chance of succeeding, so chance of total system failure is exponentially small in the number of components. Not: oh if this layer fails, the next one will catch it... which makes the chance of failure as high as the weakest link.

Reply on news.ycombinator.com