I don't understand the approach that Tesla and others are taking to self driving. Why are they not building a model of the world and then basing decisions off of that? Roads do not magically change, so why are they constantly re-evaluating things and creating a new model all the time? There are static objects in the world (curbs) and then there are dynamic ones (other vehicles, people, traffic cones when a lane is shut down, etc). Every time I see tesla self driving stuff it looks like the internal concept of the world is wiggling and vibrating and changing alll the time. It seems to me that with enough sensors and fusion you can determine that a car is in fact parked and not wiggling. You can use known maps of the land to infer roads or at least general shapes of roads that could serve as a basis to stack realtime analysis on top of it. I can't help but think going all-in on image analysis is the worst approach possible for this sort of thing. Just because we have eyes and use them to see doesn't mean that computers should too. We have rules hard coded in our minds that help us do most of driving and our eyes are just giving the extra realtime data on top. I know that the road I am on is not going to magically shift/move. You can see very very far ahead in a car and immediately make concrete decisions about your journey (I know at least for the next 500 feet I am going to be on this same road that isn't changing, all I need to do is be mindful of vehicles in front of me).

I am not articulating this well but it has been on my mind for a long time. I can't help but think the current approach of throwing GPUs and OpenCV at driving is a fools errand. Tesla eliminated a lot of their other sensors like lidar and whatnot too, to go all-in on image analysis. Foolish.