I can't understand why instead of using all the possible senses: vision, auditory, radar, lidar they want to only use vision as if there's some sense of "purity" in doing that.
An autocar seeing something that looks like a red light and breaking (or breaking for a bit and then continuing after deciding it was not a red light) is going to be acceptable to people because that is how they behave. Phantom breaking due to some radar or lidar input that the human can't perceive is going to be interpreted very negatively. I imaging auditory might be added at some point, if it is not already being used.
1) They have hundreds of millions of cars sold already with cameras that they can up-sell this to. That means more revenue. But more importantly, more training data. Massive amounts of it. Training data is the real value here. Radar/lidar/etc. might be able to detect better what is obvious to a human just looking at a thing. But given enough training data, a machine learning can probably replicate that capability so you don't actually need the fancy sensors. Adding new sensors to the mix would set them back quite a while on that front.
2) Simplicity. More sensors means more complexity integrating all the signals and gathering the right training data. More failure modes, etc. It probably also means more compute power needed to process all that data. More complex testing, etc. Scaling by keeping the sensor platform simple is a good move here.
3) The hard part of autonomous driving is actually interpreting visual signals in complex or unusual/rare situations. Roads are designed for humans with eyes. Lidar sees a blob, radar detects a pole, a camera sees a traffic sign, road markings that mean something, etc. It's a much richer signal. All the important stuff on roads is clearly visible. So, cameras are far more important for this than lidar/radar. Those are really great for avoiding crashing into things. Not so much for interpreting and classifying those things. And Tesla seems to be doing pretty OK with not crashing into things. Mostly, the amusing edge cases have to do with misinterpreting visual signals for which radar and lidar are probably not that relevant.
It's an interesting approach that they clearly believe that they can make work. It does not actually stop them from later adding more hardware to enhance things if they decide those things are needed. But it's quite interesting how far they are getting with just cameras.
The other two sound like good reasons though
Obstacle detection with lidar/radar is only interesting if you assume that object avoidance is a problem with camera based obstacle detection right now. There are lots of incidents with Teslas but I don't recall them running over pedestrians or crashing into vehicles a lot. Mostly incidents are about misinterpreting visual signals; not about crashing into stuff. If anything, their safety statistics are pretty good when autopilot is on. The cars still do dangerous/illegal/misguided things due to misinterpreting of traffic situations but it's then smart enough to get the driver out of trouble before bad things happen.
All Hail Elon.
That said, the computer vision is still far from human level. It lacks a common sense around how the world works.
You'd be able to tell there's no object anywhere near the car and disregard the light.
Lidar would be able to tell you if the object should even be evaluated.
A light hanging from a cable would still get a lidar ping back. The moon would be a void with no lidar ping.
Maybe is not acceptable (and blind in the rain/fog), so you must rely on vision to make the decision – which is why Tesla goes straight to the heart of the problem.
They haven't solved it yet, obviously, but neither has Waymo or anyone else. Tesla is probably collecting more images of 'moon or traffic light?' images than the rest of the industry combined already.
In the above video Andrej Karpathy claims that LIDAR required previously compiled maps, the creation of which is not scalable. Do you disagree with that point of view?
Without the maps, you'd need to rely pretty heavily on vision to do the right thing.
What LIDAR gives you is a very accurate point cloud to work from. You can get a point cloud from vision (and indeed, that's what telsa is doing) but it won't be near as accurate as LIDAR is by default.
Their automotive gross margin is 28% (Toyota is at 17.76%). If one automaker could afford LIDAR, it's Tesla.
That said: - way cheaper - more reliable - less power hungry - simpler and easier to debug.