It's because they're trying to code the problem instead of solving it end to end. As you can imagine if you're trying to code driving scenarios you're never going to finish. Only Tesla and Comma.ai are on the right track. Think Alpha Zero vs Stockfish.
This problem has to be solved end to end. It can be done with just vision as humans can drive with just vision. It's literally just a software, machine learning and data problem. Just takes time