Self-driving cars may have a faster reaction time, but they will never reach the level of human awareness of their surroundings while driving.
Let's see a self-driving car navigate through a construction zone, watch for instructions from a police officer who is directing traffic, or stop when kids are playing baseball in a yard and the ball rolls across the street. Answering, "well they'll have that capability someday" isn't a very compelling answer. Truly self-driving cars are dependent on technology that simply hasn't been invented yet.
How nice it will be sitting behind a fleet of self-driving cars dragging their asses down the highway at exactly the speed limit, or slamming on the brakes when a leaf flies in front of the sensors.
In addition to that, you think masses of people will be silent while losing their jobs because these robot overlords are taking the wheel?
Source: I work for a self-driving car startup.