I find this very hard to understand, especially when you're making the choice on behalf of others. Would you drive your children to school, knowing that you're making them less safe, just so that if they do die it's your fault rather than that of some programmer?
But perhaps the most convincing argument is that even the drunk, sleepy or sloppy driver at least takes a reciprocal risk in traffic. The one thing that makes me believe that people try their best to avoid hitting me is that we will both be in the accident.
Simply put: we are sort of fine with being put at risk by human behavior. That's part of of the risks we have already accepted. This is a new risk and even if it reduces another risk (fewer human drivers) we classify it as a "new risk" and we are risk averse.
But if you've already stipulated that the driverless cars are safe than driven ones, then that fact is irrelevant. The program has already been written and if it is in fact safer than my own driving then I would want to use it.
NASA is known to define a set of strict and absolute mission rules before the beginning of a mission for just this reason. See for example the rules defined for various Apollo missions [1]. Note how those rules define all sorts of limits and abort triggering conditions. This is done because there is always a desire accomplish the mission even when something has gone wrong.