In other words, just because we know that ultimately (possibly in 20+ years) the Tesla compute-only approach will be simpler and more effective, Tesla might not survive to see this happen. Instead, manual feature engineering and hacking can always give temporary gains over data and compute driven approaches. The bitter lesson was clear about this. I suspect Waymo will win, and at some point in the future once they are out of their growth at all costs stage, they will transition into their maximum value extraction stage, in which vision will make significantly more economic sense than LiDAR. But once they win, they’ll have plenty of time to see the bitter lesson through its ultimate consequences. Elon is right, but he’s probably too early.
The Bitter Lesson has held up in a lot of domains where injecting human inductive bias was detrimental. Adding LIDAR for example is not inductive bias - it's a strictly superior form of sensing. You won't call a wolf's sense of smell "hand engineered features" or a cat's reflexes a failure of evolution to extract more signal from an inferior sensory input.
Waymo will win because they want to make a product that works and not be ideological about it - that's ultimately what matters.
IMO, this is the best public description of the current state of the art: https://www.youtube.com/watch?v=92e5zD_-xDw
I expect Waymo to continue to evolve in a similar direction.
Tesla is being leapfrogged by competitors across the auto industry. All it has is first mover status (charging network).
Tesla purposefully limits the capabilities of its self driving by refusing to implement it with sensors that go beyond smartphone cameras.
My belief is that Tesla doesn’t want to actually deliver a car that can drive itself because the end result of Waymo is that fewer people will need to own a car and fleets of short term rental self-driving cars won’t spend frivolous money on prestige and luxury like consumer car buyers. They won’t lease a car and replace it every 2-3 years like some car owners do just because they like having a new car. Fleet vehicle operators purchase cars with razor thin margins and make decisions based solely on economics, as well as having a lot more purchasing leverage over car manufacturers.
I don’t think Tesla ever wants self driving to work, they just want to sell the idea of the software.
The only level 3 certified drive system available in the US (Mercedes) utilizes LIDAR.
The only operating robotaxi service also uses it.
I would say it’s the opposite of evident that cameras are enough.
I expect both to keep radar and lidar for quite a few years, but I think its usage will consistently decline to zero or near zero over the next 10 years.
The lidar didn't necessarily see the child, but it didn't have to.
A real test of lidar would have been perfectly transparent obstacles. It wouldn't have been realistic, but neither were the other tests.
There are two things that will drive companies to eliminate it:
- Cost - Even though the cost can come down quite a bit, the current multi-lidar systems they add both up front cost and maintenance cost.
- Complexity - They don't do the same things that cameras do, and can in fact never replace cameras. This means that they are not 100% fidelity backups, and therefore the decision of which sensor to be used needs to be made. A Mobileye like solution is likely the best answer here, with a discriminator network to (hopefully) accurately judge which sensor to believe for each detected point, but this is not trivial. In fact, just getting this right may be nearly as hard as the driving policy tasks themselves.
I still think lidar may stay around for special purpose tasks (eg, curb detection, or low obstacles), but the era of them being relied on heavily to detect VRUs or other vehicles will surely end.
> A real test of lidar would have been perfectly transparent obstacles. It wouldn't have been realistic, but neither were the other tests.
LiDAR would fail that test, which is why you use thermal imaging to see transparent obstacles. Roboticists have known for a long time that orthogonal sensors are the key to building robust systems that perform in a wide variety of environments.
> - Cost - Even though the cost can come down quite a bit, the current multi-lidar systems they add both up front cost and maintenance cost.
I don't understand why you think the LiDAR system is more expensive. The only evidence you have for camera based systems potentially ever working is that if you squint really hard, eyeballs seem kind of like cameras, and people drive with eyeballs, therefore AI should be able to as well. But you're not factoring in the cost of the AI as part of that system. Your proof of concept requires a human-level intelligence being in control of the car. How is that cheaper than LiDAR? Both in R&D and in the amount of resources it would take to run said AI on the car?
> - Complexity - They don't do the same things that cameras do, and can in fact never replace cameras.
Again, how is it less complex? If you need an AGI in the mix for it to even get up to the level of systems with LiDAR, how can it be simple?
> A Mobileye like solution is likely the best answer here, with a discriminator network to (hopefully) accurately judge which sensor to believe for each detected point, but this is not trivial
Huh? This is not typically how robots are made, you don't choose which sensor to believe, you construct a belief and that's informed by various sensors, each of which produce data with some sort of error that accounted for. State estimation allows us to take very untrustworthy and error-filled sensor measurements and produce accurate beliefs. This is why orthogonal sensor modalities are so important to robust system design.
You talk about minimizing complexity, but who cares about complexity? What people care about is robustness. Robust/complex solutions are preferred to brittle/simple solutions when human lives are on the line.
This is the core of the problem. Lidar doesn't actually change any of that, it still needs AI that's just as strong, if not stronger. All it gets you is a more reliable point-cloud (in some situations, maybe less reliable in others), that you'll then have to interpret.
There's no getting around the need of having a powerful AI behind it all.
Isn't this effectively a refutation of the "bitter lesson"?
Wayno is brute forcing the problem with hardware. They use Lidar.
Elon Musk's argument against Lidar is that humans only need two eyes and therefore stereoscopic vision is enough.
"Human drivers use two eyes, therefore self driving cars need two eyes." is exactly the type of thing the bitter lesson is warning against if you stretch the analogy to hardware.