The trouble with using neural nets for safety-critical real-time systems is that it's really hard to do the necessary level of validation. You can't accurately predict how the system might react in totally novel or unexpected situations. Which isn't to say that human drivers handle those situations well, but most of the time they don't do something totally bizarre or dangerous.