Various kinds of medicine also save many people, while killing small few who are unexpectedly sensitive to it. Does not mean we should abolish medicine altogether, right? Same goes for autonomous cars, after they are proven to be safer by average.
A potential ethical difference is that self-driving cars are more likely to kill non-drivers than medicines are likely to kill non-patients. The well-publicized pedestrian fatality involving Uber's self driving suite is an example of the externalized risks inherent in self driving vehicles: the side effects were fatal for someone who did not consent to the risks of self driving cars.