To the extent their actions delay the development and deployment of self-driving vehicles, they're responsible for deaths due to bad human drivers (since that's what self-driving vehicles are replacing). The faster we get self-driving vehicles, the better.