I'm worried about a time when the _statistical_ safety of AI driving is better than humans across the board, but there are horrifying correlations between AI induced accidents, like "Children on tricycles are not well recognized" or "slight fog in Seattle induces 150 car pileup". Or, AI causes worse accidents less frequently than humans. Whereas humans have lots more fender benders and minor injuries to older adults.
Not saying this is true, but it's a scenario I think about a lot.