One of the big challenges is that most self driving stacks have an interface between the perception and planning stages that is specified to be a 3D model of the world. LIDARs are particularly helpful at creating 3D models because that is essentially their native data product. So, for "traditional" AV stacks that use this interface, LIDARs are bound to improve performance a lot.
If you use a different approach, say pure imitation learning off of sensor data, you might find that LIDARs are not as important. (intuition: Humans drive well without understanding super accurate positions or velocities of objects.) Though Tesla isn't taking a pure imitation learning approach (yet), they are more in line with this strategy.
All this said, I don't think that having or not having LIDARs is a major factor in the progress of self driving. It's just a way to use money to improve perception performance (and reduce data labeling cost!) but neither is the major blocker for the industry. If we extrapolate from the last 10 years of progress, it seems like high-level self driving is going to take a while and I think that it's likely that LIDAR prices will have fallen dramatically by then and we'll see them as part of the overall sensor constellation on most vehicles.
But still, we're a long way from vision based sensing being good enough for reliable self driving so why make it difficult for yourself?
If you own a taxi fleet, the cost structure will favor spending more on Lidar if it gets you to a level 4 or 5 result faster. Cars designed for personal use will tend towards seeing the feature as a consumer "add-on" and prefer cameras. Tesla's business has focused on this latter path, trying to capture high end users first and then mainstream the results.
But if Seba is correct and we take a logistic curve pathway with both EV and self-driving tech, the cost curve will price robotaxi fleets underneath all forms of ownership within just a few years; you get much better utility from your sensor investment if the same car is making a dozen trips every day, and the consumer pays "for what they use" instead of having an unused hunk of metal taking up parking space. At that point, adoption shoots up and the consumer add-on model doesn't have a leg to stand on. Camera tech might still get better and replace the sensor package, but the race, such as it is, would be won by whomever deploys a fleet at scale first.
Mobile phones and laptops are the obvious main reason for this. Optical tracking (cameras) have economies of scale that LIDAR and RADAR can't even come close to. LIDAR and RADAR are niche features in a specific set of high priced luxury goods while optical tracking is dominating everything from government/consumer surveillance to camera phones to sports motion capture and many, many more applications.
Optical tracking has so much pressure on the field to drive costs down and to increase feature sets (innovate) that RADAR/LIDAR don't. It's going to be this way for quite a long time, so I don't really see the costs for RADAR/LIDAR getting under control anytime soon.
EDIT: Optical tracking will be the dominant method of self-driving cars, I'm almost sure of it. The trained datasets have a huge advantage in this regard. Still, it's obvious that LIDAR/RADAR will have additive value down the line. It is just very hard to see how it becomes the primary technology. This is echoed in my field as well as many others - where RADAR dominated, machine learning / software + good enough optical tracking took over at 1-3 magnitudes of cost savings.
This doesn't track for me. What was the price of the cheapest Velodyne LiDAR unit 12 years ago, and what is the price of the cheapest today? My not-in-the-industry searching says a Velodyne unit cost $75,000[1][2] 12 years ago. IIRC, estimates of Google's inhouse LiDAR sensor (which is not for sale) was about $10-20,000 per unit - this was about 5 years ago. Currently, Velodyne is selling(?) the Velarray H800[3] solid-state unit that had a $500 price-point target[4][5] during development. How does this square with your assertion that the cost of LiDAR has barely come down?
Edit: I checked, it was precisely 5 years ago and Google claimed[6] that it cut the cost of LiDAR by 90 percent! I think you were wrong to say LiDAR costs have barely come down.
1. https://www.latimes.com/business/la-fi-hy-ouster-lidar-20171...
2. https://arstechnica.com/cars/2020/10/the-technology-behind-t...
3.https://velodynelidar.com/products/velarray-h800/
4. https://www.forbes.com/sites/samabuelsamid/2020/11/13/velody...
5. https://www.reuters.com/article/velodyne-lidar-tech/velodyne...
6. https://www.businessinsider.com/googles-waymo-reduces-lidar-...
What we will likely see is that lidar equipped vehicles gain a lot of ground at first, but have to be rolled out slowly both due to cost and how they are trained. Tesla’s vision based fleet could get an update tomorrow that theoretically was able to drive almost anywhere.
Of course, Tesla has clearly been finding out that while vision is in principle sufficient, the brains behind the cameras are not so simple. In my opinion it’s still too early to call who will win in the end.
I know that other vendors have a hard time putting the necessary fast storage in the trunk and get the data off with 100G cables in the garage.
Andrej Karpathy has described this system in various talks including on Tesla AI day.
Tesla and Comma are already making profit with great margins that they needed a lot when they started developing self driving capabilities. Waymo is using Google's coffers, but so far they lost billions of dollars that they have to get back.
Regarding Lidar I wouldn't be surprised to see it in the Robotaxi that comes out in 1-2 years from Tesla, but maybe it would be a PR nightmare at this point.
Cameras don't work well in bad weather either.