I could see that causing some secondary serious problems.
Could potentially amplify its impact by combining it with the GPS to ensure it occurs at a high speed and in busy traffic too.
Most older infotainment systems have a physical power switch which is a hard interrupt to power-off. It is in my view a design mistake to have full touch-screen interfaces on infotainment.
To be sure, that when I turn off a device, it is really off.
In terms of overall fatalities, distracted drivers on their phones can be expected to continue to kill far more than vehicle hackers. It's troubling though that hacking vehicles' brakes could allow for targeted attacks with little chance of being caught. The same applies to insecure medical solutions.
The real problem in these cases is internet-connectivity + software-control. Without internet-connectivity, your primary failure mode is random chance. The probability of failure of each system is largely independent from the others as there are few ways to induce simultaneous failure in all such systems. This means that the worst-case failure mode of any individual error is the worst-case of single-system failure which, for most systems, is relatively limited. However, once you add in internet-connectivity, you now introduce an easy way to induce simultaneous failure. Since these systems will all be running the same software, an error in one is an error in all, and you can now exploit the error in all simultaneously. This instantly adds a gigantic single point of failure whose failure mode is critical failure in all systems at the same time.
Suddenly, the software failure mode of an internet-connected gas stove with remote turn-on [1] is no longer one house blowing up in the middle of the night killing possibly 4 people, it is every one of the houses buying the 3.5 million gas stoves sold this year [1] blowing up killing possibly 14 million people. The introduction of internet-connectivity raises the worst case failure mode from a tragedy to approximately a nuclear first strike and surpassing Hitler as a mass murderer. They should not be releasing internet-connected gas stoves unless every engineer on their team is comfortable that their security would be good enough to connect the US nuclear arsenal to the internet since the worst-case consequences are equally bad.
[1] https://www.geappliances.com/ge/connected-appliances/ranges-...
[2] https://www.statista.com/statistics/221447/gas-range-and-ove...
You've made an assumption about the failure pattern. Perhaps thousands of stoves could fail at once, all encountering the same defect. Something like this could happen if the software fails after 10,000 hours of continuous operation, say. (A bug of this sort once resulted in the deaths of 28 US soldiers, when a missile defence system failed. [0])
> The real problem in these cases is internet-connectivity + software-control
I agree this is a very obvious and severe problem, and it's rather shocking that it's even legal to do this. Avionics software is famously strictly regulated, but the software in road vehicles isn't.
> Without internet-connectivity, your primary failure mode is random chance
It depends. There could be a failure after some number of hours of continuous operation. Perhaps a particular stretch of road could be dangerously mishandled by all self-driving cars of a certain model. There's also a risk that deliberately crafted fake road signs could cause the system to drive dangerously, opening the door to targeted attacks.
> once you add in internet-connectivity, you now introduce an easy way to induce simultaneous failure
That's a good point. It's similar to how we place enormous trust in Windows Update, except we're talking about a life-and-death system.
> They should not be releasing internet-connected gas stoves unless every engineer on their team is comfortable that their security would be good enough to connect the US nuclear arsenal to the internet since the worst-case consequences are equally bad.
Well, no, they plainly aren't, but I do agree with your general point.
I wonder if we'll see any kind of general software regulation (that is, not specific to a domain such as aviation) to cover this sort of recklessness. Currently, the company behind such a product would only suffer after the fact, were something to go badly wrong.
[0] https://www-users.math.umn.edu/~arnold/disasters/patriot.htm...
The main point I was trying to make is that mass failures that occur with a degree of simultaneity where the failure can not be corrected between failures are relatively rare outside of internet-connected software due to an access problem. Even the cases you mentioned are generally less dangerous because you can stop usage when an error is detected. If you discover that after 10,000 hours of operation the brakes on all of your cars stop working while driving, it is unlikely that everybody will encounter this problem at the same time. A few will encounter the problem first which will let you detect the error and then work to rectify it or recall your cars before it becomes a truly catastrophic error. It is a truly rare problem where a mass produced product contains a catastrophic defect that will affect a large percentage of them simultaneously. The only ones I can really think of are drugs with long-term unexpected side effects or a response to an equally massive cause like the Carrington Event coronal mass ejection which blew out the electrical grid in 1859.
I'd count the mass ejection as a natural disaster rather than the failure of a system, but the harm is just the same of course.
Related to your medicine example: bad dietary advice could be similar to this. We're straying some ways from technological systems here, of course.
Now I'm curious. Does anyone know the manufacturer and model number for a microwave for which this is true? Have we really replaced hardware thermal switches, fuses, etc. with firmware?
It's a (perversely) interesting creative exercise to think of catastrophic consequences that might follow from the failure of a system we don't normally think of as critical. Let's restrict it to physical systems, excluding concerns like being banned from Gmail, and issues like misinformation and extremism spreading through social media.
• A bug in a sound driver could cause hearing damage, especially if someone is using earphones
• Perhaps a bug in a graphics driver could be dangerous to people with epilepsy
• A bug in a laptop's battery-management system could cause a fire on an airliner
• A bug in an overclocking tool, or in motherboard firmware, wouldn't typically cause a fire, but perhaps
• We know that bugs in satnav systems can cause inattentive drivers to cause serious accidents
• A bug in an app about food intolerances could have severe consequences. A graphics/rendering bug, perhaps in the OS, could cause the app to misrender.
AFAIK, the magnetron power supply is physicaly connected through multiple door switches, so this is an actual unoverridable and redundant disconnect. One switch shorts out the supply when the door is opened, so that if the others fail, it will still blow the fuse and cut power. I'm pretty sure that design is due to regulation, so I don't see it ever being replaced by software.