I'd argue safety is not a concern. There are a lot of "safe" things we could do, but don't. A significant percentage of the country doesn't even vaccinate its children. Self-driving cars aren't suddenly going to make us aware of our own mortality in ways that life itself hasn't already.
The real fear is lack of accountability. If a drunk plows into a crowd of pedestrians, he will be dragged out of it and (metaphorically) lynched. Justice makes us feel better about circumstances beyond our control. If North Korea test-launched an ICBM that erroneously hit Japan, we'd declare war over their typo. But when a self-driving car erroneously mows down pedestrians, we're told to just accept it, nothing we could do, mumble-mumble-training-data, and tragedies like this happen so we can be safer.
Nothing is going to condition us to resent the idea of Safety more than having our personal agency and sense of justice taken away from us in its name.