Hacker News Reader

Comment by WalterBright | Hacker News Reader

> if you’re a software engineer and you’re working on safety-critical systems, you absolutely must do due diligence and implement proper validation and testing, otherwise you could be putting human lives at risk.

I'll be blunt. This is the wrong lesson.

The correct lesson is:

Assume the software will do the worst thing possible. How will you prevent that from causing a catastrophe? I.e. make the Therac-25 physically incapable of generating an excessive dose.

This is how airplanes are made safe. Not by imagining you can create perfect hardware or software, but by assuming any part can fail, and how does the airplane continue to fly safely?

View on HN

> A malfunctioning alarm clock probably isn’t going to directly cause a person’s death, but it definitely has the potential to cause real-world harm.

When I absolutely have to get up on time, I set two alarm clocks. When traveling, I'll set my phone, and also arrange for a wakeup call.

I learned about having an independent backup from working at Boeing engineering.

That was my thought, too. Day to day, my phone alarm is fine (and I damn well tested it thoroughly before actually relying on it), but if anything life-and-death is scheduled, I'll use at least one independent extra device. And all this even though I always wake up on time and never am late. Likewise, the phone call problem has an easy solution: Assign different sounds to different callers; low and unobtrusive to the unwanted ones. And the delivery: Never trust such a thing without sufficient fallbacks. If you want safety and full control, organise the thing yourself. All in all, I get a vibe that the poster may be a wonderful coder, but really hasn't got the paranoid mindset required for mission critical work.

> Assume the software will do the worst thing possible.

That's certainly a valuable lesson when there's hardware and software involved, and the consequences of the software screwing up are potentially lethal.

But what about the case of the home depot shopping cart, or the iPhone alarm? In those cases there's only software. "Assume the software will do worst thing possible" doesn't make sense.

Decide what the worst case scenario with the phone is. If that scenario isn't a problem for you, then don't worry about it. If it is, like you'll miss that flight, have a backup alarm.

> This is how airplanes are made safe. Not by imagining you can create perfect hardware or software, but by assuming any part can fail, and how does the airplane continue to fly safely?

Boeing disagrees with you.

Tell me how.

Likely referring to the 737 MAX crashes which were ultimately caused by cost cutting by Boeing leadership and the assumption using a single sensor was sufficient.

The MCAS system should have looked at two sensors.

However, there was a backup:

1. restore trim to normal with the thumb switches

2. turn off the stab trim system.

This procedure was followed by the first MCAS incident, and the airplane continued and landed safely.

The second incident, the LA crew restored normal trim 25 times, but never turned off the stab trim system and crashed.

After that crash, Boeing issued an Emergency Airworthiness Directive, which said to (1) restore trim with the thumb switches and (2) turn off the stab trim system.

The third incident, the EA crew turned off the trim system in a steep dive, and crashed.

That said, MCAS still should have looked at the 2nd sensor.

Reply on news.ycombinator.com