It seems to me that people are actually more likely to take unreasonable risks when they have to make these calls on the spot. We often overestimate our ability to handle difficult situations and are prone to valuing "getting there in time" more than playing it safe.

NASA is known to define a set of strict and absolute mission rules before the beginning of a mission for just this reason. See for example the rules defined for various Apollo missions [1]. Note how those rules define all sorts of limits and abort triggering conditions. This is done because there is always a desire accomplish the mission even when something has gone wrong.

[1] https://www.hq.nasa.gov/alsj/alsj-MissionRules.html

Sure, which might mean that the programmer is more careless and might cause more crashes.

But if you've already stipulated that the driverless cars are safe than driven ones, then that fact is irrelevant. The program has already been written and if it is in fact safer than my own driving then I would want to use it.