Do computers make us more safe or less safe?

>"The drone on its own was going to bomb a car. But the human noticed that there were red roses on the car, so it was a wedding couple, not a terrorist. If a human had not been involved the drone may have killed an innocent just married couple!"

>This scene bothered me. It could EASILY be the other way around: the human wants to bomb and the drone (which has better vision) notices the roses. Or there may be many other ways that a computer could be BETTER than a human

The issue with the rose isn't better vision but better contextualisation, creating a plausible narrative around what is seen that changes how to act. That's something at which humans vastly outperform machines, or even more strongly a skill machines don't at all possess. It could not be 'EASILY' the other way around, at least at the current levels of technology.

A human can always consult a machine for better raw vision or image classification, but no machine right now can substitute a human for these high level judgements. Maybe in a few decades when we have AGI or whatever, but then the distinction between machine and human is moot anyway.

Most safety standards that address software's contribution to hazards use some notion of "controllability" to determine risk. For example, ISO 26262 explicitly calls it controllability [1], and it directly contributes to how rigorously the software functionality has to be tested.

MIL-STD 882E uses "Software control categories" that determine the risk introduced by software [2, page 15-17]. Since those definitions are hard to track, they also provide a table near the end that tries to be a little more specific [2, page 96].

In other words, the safety community already takes into account the need for human or mechanical intervention when it comes to risks introduced by software.

[1] https://neweagle.net/how-iso-26262-2018-update-affects-you/ [2] https://www.dau.edu/cop/armyesoh/DAU%20Sponsored%20Documents...

Personally, I don't think computers make things more or less safe than any other tool or machine does. They fix some safety problems, while adding entirely different ones. Computers are tools like any other, they can be uses.safely or unsafely. It's the application of the tool, rather than the tool itself that is safe or unsafe.

Before anyone brings up guns, I realize they're an example of an inherently unsafe tool, they can be used safely, but this example of a tool designed for murder is not what I'm referring to. I meant more general purpose tools, which computers are. Though, as mentioned in the article, computers can be used for murder, it's not their primary intended purpose the way guns are.

Really depends upon the scope of "us", no?

For small values of "us", some bad error handling or a dodgy sensor can be fatal.

For larger values of "us", small amounts of "us" debris are so much stochastic noise.

MORAL: strive to be the one who gets to manage the definition of "us".