Tesla Live Stream – Autonomy Day [video]
livestream.tesla.com
livestream.tesla.com
- Telsa is using fleet for learning (Fleet learning). This has 3 components, a. Trigger infrasture that collects the kind of data telsa is looking for training. The rational here is we don't need massive amounts of same kind of data but needs right kind of data for training neural networks correctly. b. Data Engine which learns from these examples. c. Shadow mode where they deploy the learned model and test how its doing in real world and iterate this process.
- Drivers are themselves acting as labelers and tesla is in a unique position to take advantage of this.
- People drive vision only. Visual recognition is essential for autonomy. Lidar has much less information than vision, for example to identify the thing on the road is a plastic bag, lidar provides few points. But vision gives more data for telling this.
I would second this recommendation. He did a really good job not only explaining Tesla's approach, but also breaking down the concepts in an easy to understand way for laypeople. That is a rare skill to have especially among someone who is obviously so technically accomplished.
lidar is not required and is expensive.
[1] 2019 https://arstechnica.com/cars/2019/03/dashcam-video-shows-tes... [2] 2018 https://news.ycombinator.com/item?id=16772748 [3] 2017 https://bgr.com/2017/03/02/tesla-crash-video-texas/ [4] 2016 https://www.thesun.co.uk/news/1402992/tesla-self-driving-car...
I didn't say anything about humans vs self-driving cars. I'm arguing about self-driving cars with LIDAR vs those without.
The status quo is humans so the first question is whether it can do at least as well as humans even without LIDAR. It seems to be doing that, even if the things it does get wrong are different things than the things the humans get wrong.
The question of LIDAR vs. not is a separate issue which is purely cost/benefit. Even if LIDAR has some value, it remains to be seen whether that value is ultimately more than its costs, in the same way that we know how to make cars safer by making them weigh 8000 pounds and yet in general we don't do that because the cost is more than the benefit.
The public road right before the drive to my house has an s-turn in it. The CWS goes off about half the time I go through that series of turns just hundreds of feet from my turn. I can see the turn in the road and the guard rail ahead, the car sees a stationary object I'm driving straight towards at around 45mph. If it automatically braked with its current awareness, I might never get home or get rear-ended by people following.
We are trying to build a car that doesn't crash.
If you want a car that doesn't crash, never build the car let alone drive it. Then it'll never crash.
The point is both reduction in fatalities and increase in human quality of life. Not an impossible perfection.
Why is it one or the other? Why only a human driver or a robot driver? Could you have a human driver in charge with an accident preventing robot, getting the best of both worlds? If the human falls asleep, the robot can do trivial driving, and the human still makes the more difficult driving decisions.
The question isn't if robot car is better than an average car being driven by a human. It's if the same robot car would be safer if the human was driving it, and the robot in accident prevention mode. And I don't think that is going to be true in any near future.
We already have this and we've had it for years. What do you think automatic emergency braking is? Adaptive cruise control? Lane keeping assist?
> The question isn't if robot car is better than an average car being driven by a human. It's if the same robot car would be safer if the human was driving it, and the robot in accident prevention mode. And I don't think that is going to be true in any near future.
No clue what you're trying to say here. Driver assist features don't improve safety?
I am tired of seeing driver assists being sold as driverless systems.
Autonomous cars are one of those things that really need to be done right or not at all. The notion that human drivers will be ready and able to take control at a moment's notice is nothing short of laughable.
IMO, about the only way partial self-driving cars can work would be to delegate it to a remote building full of 'drone drivers' somewhere. Heck, I'd pay for that today, just out of laziness.
A lot of the utility of self driving cars is not just increased safety. It's allowing disabled people to be more independent, reducing the need for car ownership with (less expensive) self-driving taxis, reducing the need for parking lots and allowing that land to be used differently, etc. Do you think self driving cars should be prohibited as long as they are less safe than the standard set by human+collision avoidance cars?
I don't think that makes sense unless cars with human drivers and no collision avoidance are also prohibited. Even then I'm not sure. As a society we know that driving is not totally safe but allow people to drive anyway because it's a useful tool. I don't see why the safety standard should be even higher for self-driving cars, which are even more useful, especially for people who can't drive.
One last thing is that anything that slows the adoption of self-driving cars will probably lead to a much greater number of old, less-safe, cars in use for a longer period of time, since it's very unlikely politically that the existing car stock will be just kicked off the road, regardless of what regulations are set for new ones.
If having a human in the car reduces the accident rate, then I propose making it necessary for the human to be in the car. Are you advocating that we make roads unsafe so that disabled people be able to take taxis more?
> I don't see why the safety standard should be even higher for self-driving cars.
I see it as the opposite. The advocates for driverless cars are advocating lowering standards. For me the standard is simple. If the same car is as safe driving itself as it is assisting a human driving it, then it can drive itself. You can only remove the human if the human adds no safety in the same car. Please do not compare to "the average car on the road".
I see there are positive externalities to cheaper taxis. There are really negative externalities too, in terms of traffic, urban sprawl, and carbon emissions. But I don't think we should deviate from the topic of safety for those externalities
I disagree with those, so I also disagree with the premise that having a human in the car should be required for any amount of increased safety. I think the amount matters. And keep in mind we are only talking about safety increases from present day, you're just proposing to ban them in order to achieve a greater safety increase.
My >90 year old grandmother still drives. She hasn't run anyone over, but I imagine her reaction time isn't the best. Apparently it's common for cops to be lenient with elderly folks who are known bad drivers, since in many areas they wouldn't be able to get groceries or go to the doctor otherwise. I'd argue that maintaining the status quo (by having a blanket policy) for many groups of drivers would probably be safety theater instead of anything real.
Lastly, I touched on this in my earlier post, but if you care about deaths it's very important to consider what cars are actually being replaced in addition to individual car statistics. Imagine in scenario A, 10 million older cars are replaced by 10 million driver-assist cars, and scenario B where 30 million older cars are replaced by 15 million self-driving taxis. Assume the driver-assist car is safer than the self-driving car, which is safer than the older car. Even so, the driver assist scenario may have thousands more deaths, since the uptake rate is different. And this is without taking into account any safety-improving software updates, which are very difficult to estimate but may be very significant statistically.
A simple per-car comparison may lead to all of the negative externalities and thousands of more people killed. I really do not see how that outcome could be justified in the name of "safety". At the very least, even if safety is the first and only concern, policy makers should try to minimize vehicular deaths which requires considering things at the fleet level.
Basically, Musk's point is that if self-driving cars are statistically safer than humans, then allowing humans to drive will make cars less safe again.
Furthermore (regarding some other children here), the study that Lex and his team has done show that drivers are not less engaged while Autopilot is on (as many critics say).
You mean "if driverless cars are statistically safer than humans". It shouldn't be a surprise that driver assistance decrease accidents. But Autopilot isn't a driverless system, and neither is there satisfactory proof that it is safer than a human.
I'm not sure that human+robot would be better than robot alone. Eg. The human is in control and starts moving close to the adjacent lane without indicating. Does the robot (a) assume the human driver is changing lanes and so turns on the indicator light; (b) take control of the car and move it back to the center of the current lane; (c) do nothing and let the driver change lanes without indicating? The ambiguity in the human driver's intentions means the robot might make a decision that is less safe than if it was in complete control.
Lidar gives you very rudimentary depth and positional data, but it is precise, and reliable, and the false positives are low. Lidar is not seeing the world as a series of probabilistic phantasms that may or may not be a pedestrian, or a jersey barrier, or whatever.
Elon said: "Lidar is lame! Lidar is Lame! lame lame lame lame."
But then he later said: "We've gone over this multiple times. Are we sure we have the right sensor suite? Are we sure we have everything we need?.... No."
Lidar is a crutch and we'll see how hard Tesla is limping in a couple years when Elon is claiming They're going to have Robotaxis.
To Tesla's credit, it is theoretically possible to develop a fully autonomous vehicle without needing Lidar, but before you can deploy, you need to integrate Lidar, and it can be retrofitted as a largely independent system, necessary for it's enhanced redundancy, but not needed for informing the planner, except when the Lidar is seeing things the vision system isn't.
The presentation was a pretty good run down on various techniques and methods that are being utilized by different autonomy developers. The downside is that everything is framed as a rationalization as to why Tesla has some kind of secret weapon, when they're mostly doing conventional things, and not all the conventional things you need to build a complete autonomous vehicle.
I was waiting for Elon or Karpathy to trot out the magic pixie dust that will have Tesla's driving empty in a couple years, but that was nowhere to be found.
Waymo's safetly critical disengagements are at 1 every 11,000 miles. Fender benders happen about once every 100,000 miles, so just on that Waymo is off by an order of magnitude, and Waymo spares no expense on compute or sensors.
Is there much of anything a car can do to prevent being T-Boned?
And even if there is, I don't really see it being something that radar would help much with. If you and another car are approaching perpendicularly, you'll have an almost perfect situation to get positioning and speed information over a pretty short amount of time.
Of course I've never done anything even resembling any kind of work in this space, and I don't want to pretend to know the answer, but the claims of side radar not being necessary seems to pass a smoke test.
Yes. Stop suddenly while entering the intersection when conflicting traffic is detected. Google cars do that, and they have been rear-ended several times because of it. You can find those events in the CA DMV accident reports. Better than being hit by the cross traffic.
Not that Tesla is doing any of that right now, they most likely are not. But a lidar is not going to, either. Not ever, which is the point.
I thought the fleet learning aspects Karpathy outlines came pretty damn close to the pixie dust.
With regards to fleet learning, Tesla has been going on about it for years, but more data is not better. Better data is better. Clifornia's disengagement reports have seen criticism because they place an emphasis on the number of overall miles driven as a useful measure of progress, when actually it doesn't mean much. Companies like Zoox, after spending years doing small-scale closed course testing hit the ground running, and with only 10 vehicles and a few months of driving around San Fransisco they were able to take reporters for autonomous rides, and they took a couple dozen journalists to various pre-designated spots around SF with only one disengagement the whole day.
The nice thing about the whole fleet learning narrative is that is makes Tesla drivers feel like they're helping just by driving around.
EM radiation from UV down to radio has different propagation characteristics, and materials form different attenuators. Radio easily passes through most non-metals with less attenuation than 400-800 THz radiation (light). When I was at Tesla, I tried like heck to get people to internalize this, but the 'cameras + one forward-looking radar is enough' crowd has clearly won.
Heck, RF can shoot through mud, rain, sleet, and snow. Cameras? not so much. Although I agree that the number of 'RF pixels' one gets is nowhere near what a camera provides, it does provide some pixels when the cameras are covered in road muck.
And what about when it's dark? Are the headlights going to provide enough light for the cameras to function well in all road conditions?
What an amazing position to have been in history, even if it didn't go according to the way you felt (and imo probably are) was right.
I feel like this would be able to detect trees in the distance and other objects to orient the road/lane detection. And keep it cheap. Possibly add on to existing cars.
https://hn.algolia.com/?query=solid%20state%20lidar&sort=byP...
Where did he say this? He repeatedly said they have everything they need to drive.
>but before you can deploy, you need to integrate Lidar,
Why?
>not all the conventional things you need to build a complete autonomous vehicle.
No one knows how to make a completely autonomous vehicle because they don't exist. I'll take the billionaire tech genius' approach, whose gotten farther than anyone in this field, over your armchair opinion, thanks.
What they've done so far is a dangerous lane-assistance. From what's seen they are nowhere near waymo. Would hate for tesla and uber to ruin this for everyone else.
Lidar is inherently probabilistic too,like every sensor is. Just because the photomultiplier voltage rises and some peak detection hardware can correlate it with a transmit pulse, it does not mean that there is a "point" at that position in 3d space. It has different error characteristics to vision sure, but it is still an inference problem with corresponding artifacts to deal with. In fact I would say that the different error characteristics actually make them ideal for sensor fusion, i.e. Lidar for initial depth and vision for filling in the details.
As far as I can tell, LIDAR makes a lot of prediction tasks much easier (since you have both images and LIDAR point clouds), and is also a backup for obstacle detection to ensure that your self-driving car doesn't crash.
And not sure who you are but I trust all of the other companies who are doing this a lot more. And they are all using LIDAR.
It's a mistake to think LIDAR doesn't have these drawbacks - its an optical system at its core.
It also doesn’t have all the information to recreate the 3d environment. There’s all kinds of inferences that human drivers make that require a good model of the world, to start with. For example, a car shouldn’t brake for a small object that moves with the wind, even if it looks like it’s made of concrete, or looks like a child.
I suppose if you want to use customers as guinea pigs you do have to have a low-cost solution first, which is why Telsa are advocating it. I'm sure if LIDAR were cheaper I'm sure they're be using it.
From working with some different types of photo sensors (CCD, avalanche photodiodes, photomultiplier tubes), you might imagine that incident light from another car's emitter would be much brighter than the reflected light the lidar sensors are reading and cause saturation.
This is really important. At a certain point additional miles on similar roads in nice conditions with fair traffic won’t help anymore. The data variety is really important.
Two facts:
1. Tesla can't use Lidar because of cost and aesthetics.
2. Elon has a strong incentive to oversell their approach to self-driving.
1 - https://medium.com/@LyftLevel5/https-medium-com-lyftlevel5-r...
As it stands, Tesla's vision only system has no other backup system to tell us what is off limits.
If you have other sensors, like LIDAR, radar, and high resolution maps, then you can corroborate your predictions, so for example, if the visual system predicts a 99.9% chance you can make a safe right turn, checking the LIDAR and map will provide additional checks, even if your map is out of date. Consider:
1) camera system predicts you can make a right turn. You have no map, so you make it. 2) camera system predicts you can make a right turn, you have a map, the map says there's a wall there. You don't make the turn. If the map is wrong, you're still safe, you just missed your turn off. 3) camera system predicts you can make a right turn, you have a map, the map is out of date and says there's no wall there, but a new barrier has been erected and the map doesn't contain it. This is equivalent to situation #1 4) Same as #3 but you have SIDE RADAR, so you sense the barrier 5) Same as #3, but you have SIDE LIDAR, so you sense the barrier
The problem is, Tesla is not only going "all in" with their "we only need video cameras", but Elon Musk was asked directly at intersections if the lack of side radar was a problem, and he again fell back on "video is all you need", but if that's the case, why even have front radar?
Clearly, video is not enough, and if you have front radar, there's no reason to argue against side or rear radar, and by proxy, against LIDAR.
All of this comes down to the Tesla is shipped with a cheap and fixed sensor suite, they can't change it now, it's too late, and so they are trying to justify their previous design decisions with overconfident sales tactics and PR, and I think this is somewhat reckless and dangerous to the entire AV industry because the more Tesla's that crash and kill people, the more all AVs will be tarnished as dangerous by the public.
Musk is acting like Tesla is SpaceX and they can afford to blow up rockets or astronauts during iteration, but these are consumer products. People will trust these things to be safe, not to be "beta"
Musk swept this simple fact under the rug when asked during the press conference, again overselling a 100% visual solution.
I'm not saying Waymo can't handle it, but there is a pretty big difference between being able to handle Phoenix and its suburbs compared to Boston and its suburbs for example.
So their autopilot sensors, disengagement data, and algorithms from their autopilot cars aren't being used at all for their FSD program whatsoever?
> Tesla hasn't rolled out FSD publicly so it is unclear whether Tesla would have any data about it performs in Phoenix.
Isn't that because they are using their autopilot program + the people who upgraded to the FSD tech sensors to train their FSD tech under the previously mentioned autopilot program?
You can’t even mention Tesla in the same breath as Waymo.
> But the system still wasn’t at feature parity with the first generation Autopilot, primarily because of the lack of Automatic Emergency Braking (AEB) feature. Electrek has now learned that the important feature is going to be released this week.
> As we reported during the release of the 8.1 update, AEB has been a high priority feature for Tesla, but due to the nature of the safety feature, the company needs to make sure to get the false positive rate as low as possible in order to implement it safely.
> Unlike with the first generation Autopilot, Tesla has to build the feature from the ground up using its own ‘Tesla Vision’ technology and the new sensor suite.
I think every place on Earth gets just about the same amount, ~182.5 days, of daylight in a year, right? (Not accounting for solar eclipses, or that the earth's spin is slowing)
Phoenix having relatively sunny and dry weather is a point taken though.
For example on a country road near my house, the road veers left while a side street continues perfectly straight. The current auto-pilot has a very hard time staying on the main road and almost always disengages before putting enough left-steer to stay in lane.
A human who didn’t know about the road (no knowledge of the left bend ahead), and who missed the big yellow “left curved arrow” sign, would likewise probably end up going straight onto the side street.
This is a hierarchy of stateful awareness which I believe is essential for successful self-driving — the only way to increase NN weighting to the point where the car will convincingly execute the left bend to stay on the main road is some awareness that a left bend is upcoming. This is a case where you can’t drive the road by staring at the lines. And there are many, many others.
Can you elaborate on what you mean here? Do you mean that Tesla will only be using Computer Vision for autonomy? Doesn't or wouldn't Lidar compliment CV? Or is there a practical design consideration that makes them mutual exclusive?
Humans have far higher resolution sensors and the most advanced computer ever seen behind them. And even that fails at a far higher rate than we want self driving cars to be.
The thing we most bring to driving is navigating complex, low speed environment changes - but we're not great at that either (see the number of toddlers run over in their drive ways, for example).
https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...
Thanks, but did you mean to say that people fail at a far lower rate that we want self driving cars to have? Maybe I misunderstood your point?
Expanded version of the argument: "Karpathy says 'people drive vision-only' and apparently intends that to convince us that vision-only is good enough. But (1) those people driving vision-only are using human brains, whose abilities we have not yet come close to duplicating, and (2) even with the astonishing abilities of the human brain, those people driving vision-only make a lot of mistakes and have a lot of accidents, and we want self-driving cars to be much safer than that. So the fact that people drive vision-only is no reason to think that self-driving cars should do likewise. They're trying to be safer than people, with less computing power; why shouldn't we make up for that by giving them extra sensors?"
This is really not true. The phone in your pocket is just plain better. More total pixels per unit area, much greater dynamic range, able to sample over time periods an order of magnitude faster.
The reason your vision seems better is because your brain is amazingly good at synthesizing a picture of the world around you. But all that data (the sphere around you is something like 150 MP at eye resolution) is an illusion. You're only looking at a million pixel(-equivalents) at once, thereabouts.
[1] Your eyes come close only in the middle half a degree of your fovea. Everywhere else your brain gets a blurry mess and has to extrapolate.
Your statement on phone cameras being better than the human eye isn't true today, though.
The human eye has FAR better dynamic range than any camera built to date, at any price point. This matters a great deal on sunny, cloudless days in a city with tall buildings casting dark shadows, for example.
Just like how the biological system of "stupid sensor, smart thinking" helps biological organisms, that's what is going to need to be done for computer vision as well.
About 1/3 of the human brain is dedicated to vision processing in some way. Think about that a second. One THIRD of the best, most powerful organic computer known is required for us to see what we see and we are still fooled by the things human eyes can be tricked by. It's going to take a lot of neural network training to duplicate that. Fortunately, the vision skills required to drive are a subset of overall vision capabilities.
Not at all true. All you need to do is take your camera into a dark room for proof. It can take useful pictures in environments you can't see. And with some manual control and safety precautions (seriously, don't actually do this) you'll note you can shoot useful photos of things that are very near bright sources like the sun where your eyes would be completely useless (and irrevocably damaged, again do not try this).
What you're complaining about isn't dynamic range, it's exposure control. A human brain, again, makes much better decisions about what parts of the environment are "important" when setting the aperture (iris). So the stuff you want to see is visible, where the camera is mostly just going to guess that the center of the frame is what you want and will routinely leave stuff over or underexposed.
Your last paragraph has the correct analysis but the wrong conclusion. It's the vision processing that makes the difference. The optical systems of a semiconductor camera absolutely are better, so a control system based on optics can absolutely be better.
No long exposures, no multiple exposures with varying exposure -- a single frame. I don't know of any 24-32-bit per channel sensors, and that's what you'd need.
We agree on vision processing. I thought I made it clear that it's a strong "backend" to vision (the brain or compute behind it) that makes it work so well, and that will be the case with good self-driving cars as well. I must have misstated something.
Vertebrate eyes don't have 24 bits of sensitivity, that's just insane. Rod cells are neurons, they either fire or they don't, can fire at most at about 10 Hz, and you have about a million of them at most across your whole field of vision. Do the math. What you say isn't even physically possible.
10Hz? Simply not true. Fighter pilots can identify aircraft shown on a screen for 1/250th of one second. Regular schmoes can see the difference between 30Hz and 60Hz video easily.
I am not arguing about pixel count of a camera vs the center of vision of human eyes.
I'm saying that... Nevermind. I've already said it several times and apparently you have a PhD in all things vision.
Dynamic range doesn't require bit depth. You can have an 8 bit sensor with 20 stops of dynamic range. You'll lose color/luminance resolution of course but as long as 255 is capturing a light that's 2^20 times stronger than 0 that's 20 stops of dynamic range.
But more than that theoretical point there are already sensors pushing well beyond what humans can do in terms of dynamic range. Apparently humans have a respectable 10 stops and sensors are already in the 15 to 20 stop range and pushing beyond it:
https://www.eoshd.com/2014/11/new-sony-sensor-21-stops-dynam...
To map it into 16bit values you can just use a curve to distribute the bit depth unevenly across the dynamic range. Older DSLRs did that to get perfectly usable images with just 10 bits per channel.
It's not even a secret. Light capture in a phototransistor[1] is a basically direct process without a whole lot of loss or wasted surface area. Those cone cells are living things and only have a little volume to dedicate to pigment chemistry.
[1] CCD's of course can do an order of magnitude better still, and those are easily cheap enough to put in a car.
Modern consumer cameras have far surpassed what our eyes can do in all the metrics you describe (dynamic range, resolution, response rate) and many others. It's not even close.
No we don't. We have vision which we then interpret with our _minds_. We have the ability to _understand_ what we are seeing, reason from it, and make decisions accordingly. "Deep Learning", no matter how big your dataset, fundamentally cannot possess this capability. Which goes a long way into explaining why Tesla's "self driving" tech keeps on killing people in situations that are trivially simple for humans. There's no way to rigorously test for edge cases in opaque deep learning algorithms, nor is there any way to implement redundancy as there is in ordinary deterministic systems.
This is a mad attempt to replace human cognition with "AI".
Why not? I’d agree that current models can’t, but what’s the fundamental shortcoming of deep learning?
Luckily, Karpathy is pretty good at solving research problems.
Karpathy could have easily spent entire day talking about current state of the art and how they have improved on it
Why would he do that? To help Uber?
Collecting data is actually more trivial part of this problem.
For Tesla, yes. For all others that’s the hardest part.
Musk was way ahead of time, again.
Oh come on - there are a lot of people good at solving research problems, but hard problems remain unsolved. To pretend driving autonomously in the real world is anything but very very hard is naive.
Tesla is betting BIG on NNs.
In the presentation they say that we have NNs in our brain. Which is why we can look at one photo of a dog, and distill enough identifying information in order to spot the breed from then on. Training sample of 1.
Curiously though, they than say that NNs don't work like our brains do, and require A LOT of training data from all angles, etc. Training sample of N.
The presentation doesn't actually show, how they plan to train a single NN to encompass all the knowledge for self driving. In particular, the looooooong tail. But maybe they'll have independent models, say based on "observed" conditions, which swap in and out ... based on more NN logic.
But at the end of the day, there is no known (to me) examples, of NNs even approaching human ability. Tesla is bascially hoping we will believe, they will be the first to show this capability of NNs.
Let's wait and see, 2020 is not that far away.
Also, we learn things outside of our cars. I ride a bike, this gives me insight into what bike riders will do on the road.
I have kids, so I know a kid on the pavement will be more risky / unpredictable than an adult.
I drove down a single track road today and met another car coming towards me. We stopped, then he started to reverse, there was a passing place behind him. Another car pulled up behind him, so I then reversed back into a field to let the cars past. This sort of thing happened maybe six times today as I drove around. Not at all surprising, not even noteworthy enough for any of the drivers to acknowledge even a flick of the finger in thanks. Just part of driving here.
a) Roadworks and a guy tells you to stop or waves you through. Or asks you to drive slowly. Or points to a direction that he wants you to go.
b) Woman on the side of the road is waving their arms about because her small child is crawling on the road trying to chase a toy.
If self driving cars can't look another person in the eye and identity their intent then there are going to be a lot of unforeseen problems.
Do you deploy that build or not?
Cameras are fine in the day, but when I struggle to see at night, in the rain, with spray from cars, glare from on coming traffic... that's where I would LOVE to have a lidar assisted heads up display. How Tesla think they can do better with just cameras compared to another car with cameras AND lidar just baffles the mind.
But the hard part is the software, and it's unclear that the added complexity of lidar + camera will allow the software to be better than just cameras.
As an example a recent study at Cornell shows that a stereo pair of cameras mounted high up behind the windshield provided similar quality 3d data to LIDAR. Search for "Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving" or the more consumer friendly "Elon Musk Was Right: Cheap Cameras Could Replace Lidar on Self-Driving Cars, Researchers Find".
Seems much easier to work on a stereo video stream that includes 2D, color, and extracted 3D features than trying to achieve sensor fusion with lidar + video. Especially if you want to make full use of color for things like brake lights, traffic signals, color/markings of an ambulance, school bus, lane markings, and other important details that are invisible to LIDAR.
Especially consider that if weather is terrible and the camera vision is so bad that only 5mph is safe. If LIDAR can see significantly better, do you allow it to drive at 50 mph because it can see 200 feet? For just social reasons it seems like driving as much like a human as possible is going to be best until all cars are autonmous.
Almost every driver has a rythym (or pattern, if you will) to the way they drive; and we pick up on this, and interact with them accordingly on the road. Even most bad drivers have a pattern to the way they drive. The more experienced you are, the easier it is to "read" fellow motorists, and safely interact with them on the road. One of the most difficult scenarios is when you come upon someone who drives with No rythym or pattern whatsoever. You have no clue what that person is Likely (Emphasis on likely) to do next.
I have no experience sharing the road with autonomous vehicles, but I suspect they will Have No rythym to the way they drive, or rather none that is discernible to a human. IMO this will be The major obstacle to overcome if humans and bots are to share the Same road at the same time.
So deep learning via neural networks is pretty much the only way to implement self driving cars these days. It's what everyone is using and is used also in FaceID and other recognition systems.
So Tesla isn't betting big on it any different to all of the other self driving companies. We wouldn't say Facebook is betting big on PHP just because they use that technology. And Tesla definitely isn't in the top tier compared to say Waymo.
Plus talking about neural networks approaching human ability is just marketing buzzwords. It doesn't mean anything tangible since we can't define our abilities in terms of models with hyperparameters.
Amusingly, if you wanted to train a vision system to recognize driveable areas, the straightforward approach would be to use a LIDAR system to measure road flatness. Watching the road go by on video as a human driver drives is not very useful for this. That just tells you about the part of the road a human driver chose to use, not about the road geometry itself. The driver probably won't use the road shoulders, but the drivable area system needs to evaluate them.
Recognizing drivable areas is what keeps you from hitting solid objects. You don't have to classify them, just note that they're not flat road.
For example, the engineer says the custom ASIC does 144 TOps for 2 chips vs the NVidia drive Xavier - does 21 TOps. Okay, well yeah I expect your custom ASIC does have a nice performance advantage over the equivalent GPU. at 3.5x advantage probably seems reasonable. Cue Elon Musk:
"At first seems improbable, how could it be that Tesla, who has never designed a chip before would design the best chip in the world but that is objectively what has occured. Not the best by a small margin, the best by a huge margin".
Mate, it's a dot product with some memory attached, and not a single detail your half hour deep dive has gone into suggests anything other than a bog standard ASIC.
"All the cars being produced right now have all the hardware necessary for full self-driving"
And this is where I'm totally lost. I want to believe! But he's lied so many times now. This man is sucking the credibility out of every engineer in the room. Don't repeat the same lie twice.
What's an example of this? His timelines are often much too aggressive for reality but he repeatedly delivers (eventually).
But also his claims about being able to automate the manufacture of Model 3 turned out to be very wrong and to do terrific damage to the company - they are/were way ahead with their technology and should have taken a less risky approach to getting that tech onto the market and taking advantage of that lead. And how did he not learn from the model X experience?
This is essentially unfalsifiable. In the case claims about FSD or a person on Mars are wrong then this statement can only be proven wrong when Elon Musk chooses to admit defeat - suffering a significant personal financial loss as a result - or when he retires.
Here's my issue with that. The on-chip SRAM is only 32MB, and the RAM is LPDDR4 rated at only 68GB/s.
Assuming a dot-product (multiply + add) over INT8 data, that's a limitation of 2-operations per 68GB/s (that the RAM moves at). Or 136 GIOPS (Giga-integer8 operations per second). You're limited by RAM, based on what I've seen in the presentation.
Unless their neural net is 32MB and fits entirely in on-chip SRAM. That seems unlikely to me...
Hmm... it will depend on the CNN. There's probably a good neural network design that would take advantage of this architecture. IE: A well recycled convolutional layer that probably fits within the 32MB (load those weights once, use them across the whole picture).
So the whole NN doesn't necessarily have to fit inside of 32MB to be useful. But at least, large portions have to fit. (say, a 128x128 tile with 20 hidden-layers is only 300kB). Recycling that portion across the 1080 x 1920 input would be relatively cheap.
I herp-derped early on, there seem to be CNNs that would make good use of the architecture. Still, the memory bandwidth of that chip is very low, I'd expect GDDR6 or HBM2 to definitely be superior to the 68GBps LPDDR4 chip they put in there.
[1] https://people.csail.mit.edu/emer/papers/2017.01.jssc.eyeris...
In the same way, to reach peak FLOPs on a GPU you better use the local/shared memory as much as possible.
""Tesla was inaccurate in comparing its Full Self Driving computer at 144 TOPS of processing with Nvidia Drive Xavier at 21 TOPS," a spokesman said in an email. "The correct comparison would have been against Nvidia's full self-driving computer, Nvidia Drive AGX Pegasus, which delivers 320 TOPS for AI perception, localization and path planning." The statement also contends that "while Xavier delivers 30 TOPS of processing, Tesla erroneously stated that it delivers 21 TOPS. "
The comparison made in the video: 500Watts for an hour consumes about 2-3 miles of range. In a city in slow traffic, going 12mph, that's a significant range reduction. So you might have a 10% improvement in range for the Tesla ASIC in low speed conditions.
Just because the Pegasus has 500W worst-case TDP doesn't mean that its average case would be 500W constant. If you scale back your code and idle parts of the GPU, you can drop the energy cost arbitrarily.
At least, that's how GPUs on desktops work. They only use a ton of power if you give them a ton of work. Write your code in an energy-efficient manner, and the 2080 Ti will drop down to 20W, or scale all the way up to 300W.
https://www.tomshardware.com/reviews/nvidia-geforce-rtx-2080...
Modern chips idle very well. With the right code, the Pegasus could be tuned to only use 100W (assuming good enough programmers). But Tesla's chip will NEVER be able to scale above 100W.
"Tesla's chip will NEVER be able to scale above 100W" okay, based on what? Tesla has a higher performance chip in the pipeline right now, and they could've used more silicon to achieve more TOPS if they needed it.
EDIT: Pegasus has 500W at 320 TOPS. Tesla's has 72 Watts at 144TOPS. Thus Tesla's chip, because it's focuses specifically on Tesla's NN pipeline, is about to get almost 4 times to performance per watt of the Pegasus and is much cheaper. Tesla's NN chip wouldn't help your video game, and Tesla isn't intending to compete in all the markets Nvidia operates in.
Theoretical TOPS which can only ever execute within the 32MB SRAM that Tesla has created. Otherwise, Tesla's compute chip is stuck at 68 GBps LPDDR4 RAM. Pretty slow.
Pegasus uses HBM2 chips at 500GBps. Pegasus will be able to efficiently compute neural networks that are larger than 32MB in size.
Tesla is making big bets about this tiny 32MB SRAM. Bits and pieces of the CNN can fit in there, but almost certainly not the entire neural network.
You're right that this is a specialized chip. But even for NN / Deep Learning inference, it seems a bit underpowered to me from a RAM perspective
Nvidia's offering can be really good, and so can Tesla's.
> it seems he didn't know about custom how custom ASICs are much more efficient at specific tasks than a general purpose computer
Yet, Musk is clearly stating his new chip is state-of-the-art. He is not underselling it.
> something like that is exactly who you want leading a company trying to change the rules of the industry they're in
You want someone in charge who does not understand the hardware he's touting?
> it's the job of the boots on the ground to exercise due diligence here
Who are you referring to? Employees and investors?
What, then, do you consider are the responsibilities of the CEO, if due diligence is not part of the job?
They don't need a multipurpose CPU/GPU, they have a single well defined task that dominates their compute needs. They built a chip to do this that is cost and power effective, and redundant for safety.
> Mate, it's a dot product with some memory attached, and not a single detail your half hour deep dive has gone into suggests anything other than a bog standard ASIC.
There you go, you answered the question Musk put forward. Grats.
> An application-specific integrated circuit (ASIC /ˈeɪsɪk/) is an integrated circuit (IC) customized for a particular use, rather than intended for general-purpose use.
Is there anything more "custom" than an ASIC?
https://en.wikipedia.org/wiki/Application-specific_integrate...
or full custom:
https://en.wikipedia.org/wiki/Application-specific_integrate...
https://www.nvidia.com/en-gb/self-driving-cars/drive-platfor...
It's important to see this event in the context of their significant demand and cash problems. Even enthusiastic Tesla investors like Galileo Russell suggested that Tesla is in a cash crunch and should raise money. Which hints at the main mystery about Tesla, why haven't they raised money yet?
Take a moment and think about why are they doing this event now. Elon is setting a stage for a capital raise, he's pitching the autonomy narrative after the Model 3 cash cow narrative failed. They're trying to convince investors (and customers) to give them money because money-printing autonomous taxi service is coming next year.
Also, think about why basically everyone except Tesla uses Lidars. Is it because they're stupid or because Tesla cannot use Lidars even if they wanted to?
P.S.: Nvidia issued a statement saying that Tesla's claims about their chip are incorrect: https://www.marketwatch.com/story/nvidia-says-tesla-inaccura...
Edit: In 2012, Waymo reached the milestone of handling 8 100-mile routes, specifically chosen to capture the full complexity of driving. I doubt Tesla is currently at that level. Source: https://events.technologyreview.com/video/watch/dmitri-dolgo...
the fact that suppliers don't like them has been a red flag for a while now
bulls and dreamers don't want to pay any attention to the very imminent problems with tesla's financials and furthermore tesla's executive record
there's a very good reason bears refer to him as fraudboy
but watching him scam taxpayers/unfortunately gullible people/govt (my fav ex. battery swap zev credits) while people fan over him and his company is pretty amazing
also the list of lawsuits...
Truth or not, failure or not, Musk moves forward the hopes and expectations, and I'm thankful just for that.
I don't see how we can blame him for taking position
this is wonderful
I don't know the context here but it feels like they're trying to ML-splain Karpathy?
"you say you can put bananas in your smoothie. just a thought, but perhaps, might I ask, do you have the capability to put strawberries as well, or are we not there yet?"
Basically (IIRC), during backprop the error difference gets ever smaller the further back in layers you go, ultimately getting "lost in the noise", making learning in the earlier layers more difficult to impossible.
I'm not saying RELU is the only option to make this work, or that it's the only activation function that provides a "fix" for the issue; I'm sure there are other ways to deal with vanishing gradients that I don't know about.
I also lack the mathematical knowledge as to why RELU helps in this manner, but I suspect something having to do with the lack of "asymptotic structure" approaching the extremes (I don't know what the proper term would be). Or maybe it allows for some form of "forgetting", in the prevention of multiplying very small numbers (such values just go to zero ultimately)?
Maybe someone else here with the knowledge can explain it better, and we can both learn...?
This is what everyone that is not Tesla or SpaceX are doing. And have been doing for a long time. If the CEO is not an engineer at heart, what are they?
I seriously doubt Elon Musk has more engineering knowledge than the people he hires on their specific fields. However, he can make pretty well informed strategic decisions if he knows WTF the engineers are talking about without taking their word – not even that, as explanations have to be dumbed down.
This is not a new thing. Bill Gates was like that (1). Steve Jobs was no dummy and had an engineering background, but not at the same level – he did parter with a genius engineer, however.
I think Musk is doing the right thing.
> Which might explain why he's a crummy CEO
That's quite debatable, I'd say. Isn't he getting results?
(1) https://www.joelonsoftware.com/2006/06/16/my-first-billg-rev...
I think it's very important to point out here that Elon does not run SpaceX. It's well known that he's a front man for that company, and that all day to day operations are run by an actual executive who does the job of being an executive. This is why SpaceX is doing much better than Tesla.
Sure, if you call burning billions of dollars a result.
If your goal is to make money, I'd follow anyone but Tesla or SpaceX's example.
Despite the fact that I own a Model 3 and own shares in Tesla I often find the critics make good points and just hope for the best anyways because I want electric cars to take over and I want self driving cars to develop sooner than later because driving is dangerous and tedious.
The SpaceX hate seems much less justified to me. SpaceX has been building launching, landing, and reusing rockets for years now. They've ushered in a new lower cost space age.
At the end Elon said autonomy is basically their entire expense sheet!
Whether or not Tesla can pull it off is obviously going to be an enormous topic of debate with haters and lovers on each side.
This is super, super exciting. I'm going to grab the popcorn and enjoy watching Tesla try. Whether they succeed or fail I admire them aiming so high, and planning so far ahead.
Lane following works effectively, but it wanders a bit. Mostly not enough to be bothersome, though sometimes it will hug one side or the other close enough to make me think about the big truck next to me. Sometimes a little late reacting to a curve, though not to the point of going across the line. Planned lane changes work pretty nicely. Identification of vehicles around the car is good towards the front and noticeably spotty to the rear quadrants. It reliably misidentifies the merging traffic lane as one big lane as soon as the line ends, and promptly drives to the right to straddle the 'middle' and then corrects as the line converges to the normal width.
Adaptive cruise control is flat out disappointing. Granted, I've never driven anyone else's automation, but it actually made me motion sick at times. I thought it did a really admirable job of handling people merging right in front of the nose of my car without jamming the brakes. Just smoothly slipped back until the right following distance was achieved. But in open traffic, when I came up on another car, it would abruptly slow down a bit too aggressively, and then fiddle with the speed a bit until it matched the leading car. If the lead car switched lanes, it waited until several seconds after the car was in the next lane and then abruptly sped up to the programmed max speed. There were times I could tell how frustrating this style was to people behind us who were not using a similar system.
Twice it braked suddenly, though not any kind of panic stop, and then immediately sped back up. Like it saw something concerning and then changed it's mind a split second later. Made my wife a little nervous after the second time, but it didn't happen again the rest of our trip.
Overall I'd classify it as a moderately drunk driver :). I let it drive us into a couple moderately uncomfortable merge situations just to see how it would deal with traffic entering the freeway, and while it didn't get us in a wreck or anything, it didn't make me feel like AP is on the verge of becoming viable FSD.
For what it's worth, this was on I-5 in between Portland and Seattle, in mostly light traffic. We have what are, in my experience, pretty easy relaxed freeway configurations. Long ramps, no ramp lights, etc.
I borrowed the P3D so I could have myself an extended test drive to decide if I really wanted a Model 3 or not. Wicked fast (which is still an understatement), I love the power but would probably choose to steer it myself for the forseable future. I wish they'd put in nicer seats (like, my legs were getting numb at one point!), just standard Recaros like everyone else would be perfectly nice. I think I've decided that I really like it but I'm going to wait for Model Y before pulling the trigger. I will appreciate the additional head space and seat height, and hoping that in the meantime they do v3 or v4 of the seats.
Some of your experiences are similar to mine. I think you have seen the improved merge handling, in prior versions zipper merging in heavy traffic had me disengaging due to getting too close to the merging car coming into my lane.
I have also experienced the unexpected brake pulse during adaptive cruise on a couple occasions and would really like to have seen some indication on the guidance display of what caused it. The delay when a car moves out of lane (or turns right onto a side street ahead of you) is definitely too long. The stop and go following however I think is extremely smooth, and I commute heavy highway traffic 3-4 days a week and love it. I think the amount of wheel-touch requests from AP has decreased significantly on the highway when speed is low / in traffic, I had maybe 2 or 3 touche requests in 20 minutes of heavy traffic this afternoon, although I may have adjusted volume and skipped tracks in between a couple times (which counts).
I wonder if the seats have changed? I find the drivers seat extremely comfortable. I know the back seats were revised since TM3 started shipping. I love the lack of a model year, they are always refining and enhancing the base hardware.
If I have a list of a half-dozen refinements I want which weren't included in the car at launch, how can I tell if a given used car for sale has them? Is there any type of master list of features/changes/dates?
IOW, despite Elon claiming for years that they're ahead because of the "millions of miles" of free data collection by Tesla drivers, it turns out once the engineers on the ground actually speak...they're actually pretty far behind on the data front, and most of the data they are getting is the same safe routes over and over again since they're not trying to put Autopilot into novel situations to acquire new data or test the ML algorithm's capabilities to handle unique/novel situations (like trucks turning in the highway, or freeway dividers).
My takeaway from the presentation is that Tesla will perform better than other companies in this space (although I don't know enough about Waymo to comment) due to the following:
-You want a large dataset (Tesla and many companies have this and can simulate) -You want a varied/diverse dataset (Tesla and many companies have this and can simulate)--the point here is simulations for simple cases work (you can only simulate when you know), but for complex ones are close to the difficulty of actual FSD -You want a real dataset (Tesla is the only company who can say this and can say they have data on how X00Ks of drivers will handle these situations)
And Autpilot's performance (including its numerous regressions) suggests very strongly either that it doesn't have a very large data set, or else that it has a large data set of everyone doing roughly the same thing almost all of the time. These are the two most logical explanations for Autopilot's tendency to veer toward freeway dividers even (especially?) after updates.
I don't thinks it is a fair characterization: first the notion of "large" is fairly subjective. But more importantly the fact that they collect data on pre-determined triggers is just a guarantee that the dataset is not over-fitted (let say to the 280 and 101 in the Bay Area and to Elon's commute in LA) and instead has good coverage of the world.
Their capabilities of triggering on situation allows them to grow the dataset quickly in a supervised way. It is all about the granularity of these triggers, imagine that you can express "collect situation in tunnels with jerk higher than X m/s3" or "collect all lane change abort in snow condition", ... In the next 24h you get data from all over the world and this data is automatically tagged and classified by the neural net.
They also have cars all over the planet while Waymo seems to only be tested in a few cities. I can't find the podcast, but Elon mentioned how one of the main difficulties is the varying types of intersections. There are only so many intersections in Mountain View. But Tesla is getting data for all kinds of intersections.
It's also pretty clear based on the selective presentation of data that they're even further behind Waymo than everyone thought. This was the time to actually prove that their ahead, and all they could give was more of Elon's hot air.
At the end Elon said autonomy is basically their entire expense sheet!
Must have missed that part, but if he actually said that it would have been a material misrepresentation of Tesla's financials, since they supposedly made a chunk of cars in Q1...OTOH, if the statement is true and Tesla basically made no cars in Q1, it would explain by Panasonic is refusing to make additional investments in the Gigafactory.
Whether they succeed or fail I admire them aiming so high, and planning so far ahead.
Other companies are both capable of aiming high, planning, and succeeding. Tesla has yet to demonstrate the capacity to do items 2 and 3 on this list. Case in point: pretty much everything about the Model 3 launch, from building to distribution, the solar roof panel (and really, the entire SolarCity acquisition). I'm even going to throw BoringCo in there since it's based out of the Tesla lot and uses Tesla vehicles and engineers.
Now it's a robotaxi company? What is it going to be next year, a debt restructuring company?
Oh come on! If even half of what he says comes true in the next five years the world will not be the same.
Entire car rated to 1 million miles (including the battery pack) with minimal servicing
Level five autonomy at the end of 2020. Remove steering wheels and pedals from cars a year after that.
After robo-fleet goes live Tesla will be "massively profitable"
Cars that today cost $38k will make $300k + in the next 11 years for their owners in the fleet
etc.
> Other companies are both capable of aiming high, planning, and succeeding
He mentioned that all their cars since 2016 have had redundant power steering, redundant wiring and even redundant power. Even if it loses the main power pack it can still steer and brake safely.
Give me ONE example of another auto manufacturer that has thought far enough ahead to design that kind of redundancy into their vehicles planning for self driving? It simply does not exist in the industry.
The man really is seeing the future and trying his best to make it happen.
Like I said there will be a huge debate about if he can actually do it.... I'm just saying that if he can it's going to be a very exciting time to be alive
I could say 20 things that if even 1 comes true in the next five years the world will not be the same!
Also note, I'm not saying Elon will do all or even half of them. I'm saying it's going to be damn exciting to watch, and I'm keeping my fingers crossed that he does, because I'd like the world to get better, and I'm happy to be an optimist.
Like what? I often hear people saying that experts were saying it's impossible to land a rocket or make a good EV car. But I didn't hear experts saying that.
What I did hear, is experts saying it's impossible to make it in cost effective way. And for that - economy for spacex is still to be proven. Tesla is still bleeding money (apart from occasional financially engineered quarters).
That being said - while I think most of his companies are doomed to fail (due to gross mismanagement), I'm happy Elon is around. He's lying a lot, and always overestimating by an order of magnitude what he can deliver. But he does inspire people to go into science and engineering - spacex landing rockets are saturn 5 or space shuttle of this generation.
Given that so many Tesla owners have complained about the power steering and wiring going back (and the delays in getting these issues fixed) the most likely explanation is that Tesla is attempting to forestall maintenance issues by simply adding built-in backups rather than by improving quality control and making the primaries work better.
IOW: having built-in redundant parts for some of the most important parts of the car--which aren't known for breaking down in cars from other manufacturers--is actually a sign of failure on Tesla's part, not a sign of forward-thinking.
Entire car rated to 1 million miles (including the battery pack) with minimal servicing
That's nice. Tesla can't manage to get car panels aligned properly, and a number of cars are still awaiting parts a year or more after they went in for service, and Tesla hasn't managed to fix its own billion-dollar factory line to work properly...but in 5 years they're promising better-made cars than any other car maker? Toyoto has earned the right to make that claim with their decades of consistency. Tesla hasn't, especially not after the Model 3 fiascos.
Level five autonomy at the end of 2020. Remove steering wheels and pedals from cars a year after that.
Call me when they manage to actually stop their cars from veering toward freeway dividers. It was still happening as of November 2018... Also, there's the pesky little problem of Tesla not yet having demonstrated a door-to-door trip yet in uncontrolled conditions despite promising that back in 2016.
After robo-fleet goes live Tesla will be "massively profitable"
See above. Need to hit Level 5 before then. But based on all the lots filled with unpurchased Model 3s, they'll probably have the fleet for it if they ever achieve Level 5...
- the Unsworth lie
- the "zero concern and I mean zero about 10k Model 3s per week in Dec 2018" lie
- the "funding secured" and "only uncertainty is shareholder approval" lies
- the "short burn of the century" lies
I mean, they were even wrong about Nvidia's chip today, Nvidia had to issue a statement: https://www.marketwatch.com/story/nvidia-says-tesla-inaccura...
[1] https://www.nvidia.com/en-us/self-driving-cars/drive-platfor...
Thanks for your link though. I couldn’t find 320 anywhere
"All we need to do is improve the software" - Musk [12:07 PDT]
LIDAR "unnecessary" - Musk [12:13 PDT]
Computer vision guy is now speaking.
Recognizes "driveable space", not just obstacles. Video shown, but just for a freeway. This is crucial to safety. Need to see this is a cluttered environment.
That's an old claim, see this 2016 press release [1]:
> as of today, all Tesla vehicles produced in our factory – including Model 3 – will have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver.
At this point I believe the claim once I see the self-driving functionality having been rolled out to the public and the accident number been reduced.
[1]: https://www.tesla.com/blog/all-tesla-cars-being-produced-now...
Sensors with non-moving optical windows can use other strategies, such as air knives or wipers.
https://www.natice.noaa.gov/pub/ims/ims_v3/ims_gif/ARCHIVE/U...
Here is Europe and Asia on the same day:
https://www.natice.noaa.gov/pub/ims/ims_v3/ims_gif/ARCHIVE/E...
For someone who lives in or close to that white part, "most scenarios" includes snow. Arguably for Level 4 autonomous cars in those areas, you either need to fully disable autonomy in September and enable it again in late April, or you'll need to handle snow.
"No driver attention is ever required for safety (...) self-driving is only supported in limited circumstances (e.g. geofencing), and when these circumstances are no longer met the vehicle must be able to safely abort the trip, e.g. park the car, if the driver does not retake control."
How would that work if you're out driving on the highway, and it starts snowing hard so the car can't see anything? Just park on the highway?
If your car can't safely handle such a scenario, the automous feature would have to be "season-fenced" in addition to geofenced.
I will simply refer back to my initial comment in this thread: "there is going to be a decent amount of time between when a self driving car can handle most scenarios and when it can handle all possible scenarios". Blizzards are not in the "most scenarios" group. Barring emergencies, no one should be driving in a blizzard let alone an autonomous car.
https://www.cambridge.org/core/journals/journal-of-glaciolog...
Because he's sold a bunch of cars (and a bunch of stock in the company selling the cars) without LIDAR with the explicit claim they are hardware-ready for full self-driving.
Freeway shoulders are not safe places to stop. You're likely to get hit by a drunk or distracted driver. Ask any police officer or tow truck driver. That's why stopping on the shoulder is only for emergencies.
Most approaches want to use a wavelength that has as little ambient light as possible. That light is absent precisely because it is absorbed by water vapor in the atmosphere. The alternative approach must deal with lower signal:noise, but is not substantially affected by weather.
Simple physical solutions are available if necessary for the cameras too. I don't have a tesla so I don't know what measures they use to keep the cameras clean but I'm sure they've thought about it and designed for it.
https://teslamotorsclub.com/tmc/threads/water-dripping-into-...
Tesla is new to the game, give them a few more cars to catch small stuff like that.
But ask yourself, what's more likely, the other car companies catch up to their AI and battery tech, or Tesla fixes the trunk in the next model?
In our 2005 DARPA Grand Challenge vehicle, we had a system to clean the camera and LIDAR. It used a spray nozzle, and alternated spraying windshield cleaner and air. This is a commercial truck accessory used on mining trucks.
Every extra they tack on to the base product is one they will be expected to deliver on as well diverting attention from the main problem they are dealing with, which may in the longer term leave open enough room for the competition to wiggle through.
Sure, autonomy is a big deal, but it is also something that will once cracked be an instant commodity and there is a lot of money and talent focused on that particular problem, which is surprisingly hard to do well.
If Tesla ends up going under because of one of the side shows (Solar City, autonomy, Power Wall etc) that would be a serious loss.
SpaceX isn't trying to just do what has always been done and just make it 5 or 10% cheaper. That's boring. SpaceX is completely turning the launch industry on it's head.
Tesla is the same. Elon isn't interested in making "just" an electric car. He wants to change the world, and electric cars that drive themselves will do that.
Can he do it soon? I'm not an expert, I don't know. But damn it's exciting to watch.
Perhaps this is the intent rather than a side-effect.
Intentional misdirection to distract investors.
Musk's ego plays a lot into this for better or worse.
Apple's work on developing their own SOC (A13) have paid dividends for them. The same legendary chip designer (Jim Keller) designed the new Tesla chip after working for Apple.
Elaborate.
Building whole process that will keep on producing solutions that are better than what suppliers can offer, especially in highly competitive area, is way way harder. That's where the real money are spent.
I get what you're saying though. But if they manage to pull off the Robotaxi fleet, then $$$ won't be much of an issue.
If anything, the solar city/roof shingles side-show has seemed the worst to me personally, alongside the falcon-wing doors and the self-infliction of twitter harm by the CEO.
Can't they patent their breakthroughs? How is this different than pharma breakthroughs?
Definitely not and I'm not sure why you think this.
The data pipeline advantage is a big part of this talk.
Not if the company that cracks it eats its competitors or, assuming it's like waymo, commoditizes car manufacturers into suppliers for autonomous taxi fleets.
If Musks ideas about self-driving actually become true, he holds the holy grail of the car industry. On one side, self-driving abilities become crucial in future car sales. On the other side, if he can make the Tesla car sharing network happen, the revenue of that could be a magnitude or more larger than the one of car selling. This would justify some of the crazier valuations of Tesla stock.
- First principles hardware design of focused self driving computer (many times better than any competing existing hardware). Already shipping in all newly produced cars. Currently working on next gen that will be 3x better to ship in a couple years.
- Lidar is an unnecessary mistake that competitors are making that won't succeed (too expensive, need too many, unnecessary).
- Real world fleet testing is critical to success, simulations are not good enough since there are too many unknown unknowns in the real world. Tesla uses simulations too, but nobody else comes close on real world fleet testing.
"next gen that will be 3x better to ship in a couple years"
applying psudomathematical varnish to marketing-speak
This is very controversial and rest of the industry thinks exact opposite, many even claiming that Tesla is being irresponsible and even delusional in trying to do autonomy without lidar. The main points I have heard in favor of lidar are that computer vision is very flaky not only in suboptimal weather but even in good weather. Cameras are simply no where close to in performance in dynamic range, rapid adaptation, focusing etc as human eye. Imagine car going under the shaded road with rapid changes between bright light and shade. The likelihood that your depth estimation will get messed up is very high. Of course, night driving becomes highly questionable as well. In additional the long range depth estimation is very flaky with stereo vision right now and a topic of research for mono-vision. If you want to retreat to level-4 only and that too with conservative speed, weather etc then may be vision+radar more doable?
Elon predicts all competitors will eventually drop lidar. He mentions it's expensive, but also not as good in a lot of cases (and all roads/signs are designed for vision).
He argues that getting vision to work is a prerequisite for getting self-driving to work and once you have it working, lidar is worthless (and unnecessary).
Case in point: both the Gigafactory (vastly overbuilt for the quantity of batteries actually produced) and the Alien Dreadnought (vastly overbuilt for the number of cars Tesla current produces...assuming that Tesla is ever able to get the fancy automation working). Boring Co digging a two-mile tunnel in West LA without bothering to learn how to pour concrete smoothly, or to make the "rails" the proper width, or learning about ventilation, or access points....
https://www.theverge.com/2018/2/7/16988628/elon-musk-lidar-s...
But only time will tell.
- If you are trying to sell the Model 3 directly to the end consumer with autonomous mode, the extra $10K for Lidar and bulk (which will greatly effect the exterior design) are definitely non-starters.
- If you are going to robo-taxi route (such as Waymo and Uber), the extra one time $10K cost to add lidar for the 5 year life of the car is probably a blip on the income statement of the operator as compared to a full time human driver which probably costs $10K PER MONTH for the lifetime of the service. For the robo-taxi business model - its a bit of a no brainer - they could stick every sensor known to man on the car and still make out like a warlord by getting rid of the human driver but maintained a 100% safety record. Plus making the car stand-out with a unique Lidar inclusive shape is a great marketing differentiator. Also reduces your liability if a taxi rider ever sues since you can claim you have redundancy in the system.
- Also only a matter of time before lidar cheap enough that even Tesla will add for redundancy and edge cases.
>Currently working on next gen that will be 3x better to ship in a couple years.
Someone asked what the primary design objective of the next generation chip would be and the engineer muttered 'safety' before Musk made the 3x claim.
Except that it isn't. NVIDA's Drive AGX Pegasus delivers 320 TOPS:
https://www.marketwatch.com/story/nvidia-says-tesla-inaccura...
Of course, nobody inside the Musk Reality Distortion Field actually cares about this.
That's ... not great. If that was CPU type of unit, it'd be great. But TPU-type accelerators are growing at massive speed (as it's still pretty new and simple tech), where you're more looking for 10x type of gains.
Every law enforcement entity on the planet: drool
The future is here, and computers will see everything in excruciating detail via multiple planes of information. The power you consume, the data you send, the money you spend, the media you watch, the words you write, where you go.
It’s all out there in the world. There’s no taking it back, though we may be able to legislate about its use.
You can't say "You can query for anything!" and have it be anonymized.
In fact, if you're taking a camera shot, that's the most broadband sensor imaginable, and no thanks, I don't trust any company with profit motive not to crumble and pop out a software update to selectively disable picture blurrers, or to not have a Boolean flag in your data scrubber.
As a company, making that statement is just trying to slip one past and hope nobody asks inconvenient questions about it.
"To me right now, this seems 'game, set, and match,'" Musk said. "I could be wrong, but it appears to be the case that Tesla is vastly ahead of everyone."
I am eager to see what they unveil today.
It was mind boggling. I am hoping Tesla can provide some specifics today because it seems Elon is living in a fantasy world (albeit one I'd like to live in if we can actually get safe self-driving cars).
also I can't reconcile how the new hardware is this huge leap ahead beyond raw computing power if, by Tesla own claims, previous hardware was perfectly capable of autonomous driving.
seems people were getting fooled either now or before.
https://www.google.com/url?sa=i&source=web&cd=&ved=2ahUKEwiR...
And Elon has a long history of making false claims about Tesla’s progress. For example in 2015 and 2016 he claimed that Teslas would be fully self-driving by 2018.
So why shouldn’t we be skeptical?
https://arstechnica.com/cars/2019/03/teslas-self-driving-str...
Your "third party research" is obviously bullshit, they go as far as including Apple in their ranking.
This right here is just typical worthless marketing press release spam from a management consultancy firm.
No one knows if Tesla strategy will work because they don't have the data collection in place.
They have no way to store or transmit the massive data you are describing off the platform do they?
My understanding is that they have very limited storage and transmit onboard.
I mean saying a phone can upload videos to youtube but a can can't to tesla is a weird ledge to stand on. Even their windshield wipers work based on sending video data to tesla to be learned on.
Your link:
> According to Electrek, Tesla trails behind other companies in terms of autonomous driving tech based on a list created by Navigant Research, an independent research firm.
Electrek’s article:
> Electrek’s Take
> I think Navigant’s autonomous leaderboard is ridiculous. There are way too many brands that keep most of their development under wraps, which makes it hard to evaluate them and therefore, it gives very little value to a leaderboard like this in my opinion.
Electrek is not exactly unbiased. It's literally called EV and Tesla news. Fanboi site's opinion should probably be taken with a grain of salt.
Here's the Navigant executive summary directly: https://www.navigantresearch.com/reports/navigant-research-l...
The Waymo end-game that I heard was "able to go through a drive-thru". I highly doubt Tesla is anywhere near that point.
(1) https://m.heise.de/autos/artikel/Test-Tesla-Model-3-4400919....
Basically, most other manufacturers like Opel, Audi, Mercedes, Hyundai, VW, Volvo, Ford, etc. has had for several years the feature to detect speed limit from computer vision recognizing the road signs. And it works reliably, as is pointed out in your link.
How can Tesla be a leader in using computer vision for cars, but not be able to read the road signs?
The kind of drive-thru that Tesla is currently associated with involves semis rather than fast food and it would be really nice to hear that they've at least licked that particular bug (and for good, this time).
Your point it very astute.
Among a few other ML/AI MOOCs, I completed Udacity's "Self-Driving Car Engineer" nanodegree - so when I'm out driving, I often come upon situations where I wonder "how would a self-driving car navigate this?"
Today, driving in to work (note: USA), I noticed one intersection I've been through many times before, and that question came to mind. The intersection is interesting, because on approaching it, the road curves to the right, and you can actually see one of the traffic lights on the left before you even see the intersection. By the time you see the intersection, you're already on top of it.
So as you round the curve, you see the lone traffic signal (red/yellow/green); if it is red, do you start to brake, or do you wait until you can "see" more traffic signals? If you wait - will you have time to slow down and/or stop? ...and so forth.
This and others are all kind of "edge cases" that will need to be trained on, and/or perhaps other cues for self-driving vehicles installed or set up so the vehicles can navigate such areas successfully. I know when I first went through the intersection it was a bit of a surprise; it's not a very safe intersection (going home in the opposite direction is not any better; in that direction, you're headed downhill, have to cross the intersection, and immediately start turning to the left after going through - the curve is really abrupt, and you have protected/unprotected left-hand turns both directions, etc).
Other than for accidents, the SEC investigation, etc.
I can imagine that police could mine this just like they're doing with Google geolocation data.
Except that it isn't, and even Karpathy said the quantity doesn't matter, it's the data quality.
Yes, two of the autonomous vehicle leaders are not using any data whatsoever.
I thought this was the smartest forum on the internet?
Notice the cherry-picked examples in the presentation. There is a whole class of problems the field cars can never help with, since they lack the dead-reckoning sensor setup and precise odometry a development car would have.
Can you give an example? I'm curious what kind of triggers strictly require lab-calibrated hardware.
If you call something "autopilot" and promote it as if it will drive for you and then it ends up killing dozens of people ... that's where successful class actions come from.
(I obviously don't want people to die either.)
If you think this is a new thing, you haven't followed the car industry much.
Just as an example, my 2002 el cheapo Peugot was bought without the option to show instantaneous MPG/average MPG/range/etc. Spend two hours soldering/gluing on the missing $2 toggle switch, ask a friend with the Peugeot Planet update tool to enable it in software, and Bob's your auntie.
It goes much further than that though, especially when you get into chip tuning. Software upgrades that add 50 horsepower to your engine output are commonplace.
But as for the "actively hostile to 3rd party repairs/mods", this part scares me to.
It's the same thing they ran into with the software locked batteries. The goal is to have minimal overhead to try to drive costs down on Model 3, and making the SR just a software limited version of SR+ actually ends up pushing the overall cost down, rather than having to set up an entirely different line that does the cloth seats and the aluminum roof.
More specifically, I'm bothered by the fact that if you get into a crash in a Tesla, Tesla can disable your car and stop it from activating unless you do repairs at their approved dealership. You can't just buy parts from a scrapyard and get it running again, Tesla is the gatekeeper and they don't let anyone else hold the keys(they don't even release service manuals unless absolutely required by law). In contrast I could buy a brand new Mercedes CLS with its very respectable smart cruise control and Mercedes can't do anything to stop me from replacing the engine, the head unit, or fixing the whole thing if it's totalled - they just can't. It's not a freedom I'm willing to lose by buying a Tesla.
I knew I'd find someone here who hates when John Deere locks down their vehicles, but is fine when Elon Musk does.
How about 2 thus far and Tesla was not at fault.
That does not mean that their cars can self drive today.
That does not mean that their cars can self drive three years from now.
It's 100% not proven or obvious how car self driving skill and car self driving error rates scale with compute -- but it's surely not linear.
Dozens of times in my life, I have been driving down the freeway at 70 MPH in a 60 MPH zone, and I've noticed someone weaving through traffic behind me, going closer to 140 MPH.
I need to know whether to change lanes, stay in my lane, stop changing lanes, pump my breaks to indicate there is slowdown ahead of me that car might not see, etc.
Just food for thought.
EDIT: I'd also like my vehicle to be good at avoiding a car that's about to T-Bone me, at night, with no headlights on. I may not be very good at avoiding that kind of accident today, but if LIDAR is necessary to protect me from that kind of accident, then I might think it's a wonderful idea.
It is absolutely something a car company should worry about. Avoiding obstacles from any direction.
If a power line is falling down. If a construction crane is swinging wildly.
It's Defensive Driving.
Also, note, I said "closer to 140 MPH". I actually just saw another one on Sunday. Crotch Rocket motorcycle, in the far right line on a 5 lane freeway. No helmet. Hey, someone's got to be an organ donor, right?
Your other examples would be happening in front of the car.
Aborting a lane change before you initiate it is changing your behavior, in response to a danger.
> Your other examples would be happening in front of the car.
Or from the side.
Or from behind, again. If I'm in stop-and-go, danger can come from behind. Picture an ambulance coming up from behind. You absolutely should change your behavior, depending on conditions all around you. Not just the front.
On the other hand, I've been sensing that Tesla is finding it harder and harder to raise cash and has been getting increasingly desperate. Are we on the cusp of a new transformative technology or the peak of the mother of all bubbles? Time will tell.
Option 2: He's lying to get money.
Compare the 2016 demo video.[2] That's a tougher route. That's the one where we now know it took a lot of tries to get a clean video.
Waymo and Cruise have put up videos of their cars in city traffic. They get criticized for things like getting stuck behind double-parked cars, and being a bit shy of parked cars that project into a traffic lane. But they get where they are going. Tesla is not showing anything near that level.
Supposedly the analysts at the meeting got to ride in a self-driving car. Anyone seen reports from them?
[1] https://www.youtube.com/watch?v=tlThdr3O5Qo [2] https://www.youtube.com/watch?v=eAal0juXXzU
These guys also went on the ride-along: https://youtu.be/2BZHXh1nbWc?t=360
Second video says the driver "touched the steering wheel once" in 15 minutes of driving. That's a disconnect. So, one disconnect in, what, 10 miles of driving? Waymo reported one disconnect per 1,392 miles to CA DMV last year.
Amusingly, the riders were not allowed to take video.
It is "deactivation of the autonomous mode when a failure of the autonomous technology is detected or when the safe operation of the vehicle requires that the autonomous vehicle test driver disengage the autonomous mode and take immediate manual control of the vehicle."
Strong words from Musk about sticking with video only.
Later on in the software talk:
“Lidar is really a shortcut which sidesteps the fundamental problems...and gives us a false sense of progress”
Doubt it.
That is point behind what is known as "probabilistic robotics" - the real world has noise, and you need to be able to deal with it. Don't expect perfect sensors, don't expect a perfect environment.
Self-driving vehicles are the application of probabilistic robotic principles to a real-world task - quite possibly one of the most difficult tasks for the field. To be quite honest, it's amazing how well it's worked in such a short period of time.
I think cameras and machine vision approaches will be needed for self-driving vehicles to be fully successful, but I wouldn't say that LIDAR should be counted out. It will probably be necessary - even required - to have all of the sensors currently being used, not just a subset.
One kind of sensor that hasn't been explored much that I think will be needed is some kind of audio input; some kind of wide-range microphone (probably binaural or stereo) to take in environmental audio and use that for driving cues. Some simple examples might be for vehicle horns, or the squeal of tires, or the revving up of an engine indicating someone is speeding up aggressively, or an emergency siren, etc.
There may even be other sensors needed or that could provide other data to fill in certain gaps of environment knowledge to help a self-driving vehicle navigate. I don't think any one or another should be discounted.
Lidar is expensive, and gives surprisingly limited data (just points of depth) and is like a crutch that will only get so far. It will not get you to full autonomy so why not spend the time and money on vision which will (as proven by humans). They also do use radar and ultrasonics (not visual spectrum).
At some point your machine must be good enough via machine learning to give a very good impression of understanding intention in other road users, pedestrians etc. One example they gave was a distracted pedestrian with a phone. Lidar tells you nothing save an obstacle is on the pavement, it won’t tell you they might step out without looking, but machine learning on a massive dataset can.
Once Tesla can actually handle those two extremely basic tasks maybe they can start talking about how vision is better than LIDAR. Until then, it's just more hot air, and that's not even taking into account Tesla's claims of using ML to predict the actions of uncontrolled independent agents in the field of view.
Starting from a purely visual domain likely dooms your system to making the same types of mistakes that humans make visually in judgement. At least with accurate maps and LIDAR as backup, you can sanity check output of your visual processing against a map and LIDAR. If your claim is that the map might be out of date, or the LIDAR too low rez, it still helps to err on the size of caution.
In the worst case, the map tells you you can't go somewhere that you're allowed to go. In the case where it says you are allowed to go somewhere were you shouldn't, well your visual system should be telling you not to go there. If it isn't, you're in a lot worse trouble than having a map with flaws.
Likewise, the claims that "LIDAR is expensive" is like claiming EVs are expensive because "batteries are expensive!" If AVs become common, then they'll be a huge demand for LIDAR and the costs will decline. LIDAR costs are already declining. Maybe if Elon took some of Tesla's miracle engineers and had them make a LIDAR, they could not only defeat Nvidia at chip making, but defeat all LIDAR makers as well. (sarcasm) My guess is the real problem with LIDAR is drag and vehicle trim/styling.
At this point, Tesla is basically stuck. They bet big early on a shitty sensor suite before AV technology had been worked out, and wave their hands about how using consumers as guinea pigs to feed them fleet data will magically fix deficiencies in sensors. Well, what if this is a false hope and it doesn't? It would mean they'd get sued by everyone who bought the AV suite as an option and have to issue refunds or recalls to upgrade.
I've said it before and I'll say it again, you don't ship AV as an MVP on a $100k vehicle and promise magic upgrades and fixes later before the technology is even close. It's risking people's lives and it's already killed people.
And they're going to kill people.
It's hard enough already to communicate to the public in any meaningful sense ML results without #SKYNET brigading, Tesla is going to make it 100x harder dragging us all through their mud.
I think at this point (of having been a matrix boi long enough) I am allowed to publicly state my belief that we are very very very far away from human-level visual perception. Just staggeringly, incomprehensibly far. We've just barely started to be able to do the most basic things, sometimes.
Is it an incredible amount of collective dunning-kruger that is blinding tesla? Or perhaps willful ignorance?
And I would secretly look forward to. I was part of a self-driving team at university and we had to raise funds especially to be able to afford a Velodyne HDL-32.
Mumbled Answer: "Safety."
...doesn't that mean the current-gen chip... isn't as safe as you want?
I worry that the laws on the books are insufficient to judge whether a computer is safe to drive. If Tesla already has plans to increase safety, I'd like to know how they judge it, where they think they are, where they'd like to improve, etc.
As a voter.
1) computer has faster reaction times.
2) 360-degree awareness
3) better awareness of vehicle performance.
Imagine a scenario where you are at speed and get a tire blowout. The car can sense it instantly through tire pressure sensor, and counter the swerve appropriately - something that humans are very poorly prepared or practiced for.
Alternatively, a driver in a neighboring lane makes an unsafe lane change into your lane. The car sees him, honks the horn, moves/brakes/accelerates to avoid the other car, before you've even seen whats happening.
Optimally safe should still be the goal. That is, no human could have provided alternative input to the computer that would have created a safer outcome.
Volvo leadership states the exact same thing about their car designs. The primary motivation “in everything they do” is safety.
The statement can be made entirely distinct from the current safety level.
Lane change on crowded freeway: https://twitter.com/hamids/status/1120472369762590722 - It manages it if not quite as smooth as a human.
Summon in parking lot: https://twitter.com/hamids/status/1120446405410205701 likewise.
There's also an official Tesla sped up demo vid https://www.youtube.com/watch?v=tlThdr3O5Qo - hard to tell really.
Same video slowed with the display magnified and some Reddit discussion https://www.reddit.com/r/teslamotors/comments/bgb9or/fsd_dem...
If they think they’ve cleared the bar just by beating the crash statistics of all cars, then they are still in the business of trading lives.
How does this compare to TPUs and the Neural Engine in iPhone CPUs?
My guess is that Tesla will need another round of financing along the way and they have to make sure that the stock's price stays up. Musk is a great promoter but at some point he needs to come back to reality.
(replay after the livestream ended)
On one hand there are hyper-bulls who claim Tesla is a $4000 stock and the future of transportation. On the other, hyper-bears claim the equity should trade around $0-$10. There seems to be no middle ground.
It seems like they are almost betting the company on FSD. I don’t think FSD is really even close to a possibility over the next 5-10yrs. I hope I’m wrong, but if I’m right, I don’t see how Tesla keeps going on like this.
IMO they don't have any choice. The more and longer they operate like a car company shipping bigger and bigger volumes, the more their financials will be undeniably trend towards those of the existing car companies and the more their existing market valuation will be hard to justify.
They need something like this to not get traded at traditional car industry multiples.
Yes! Totally agree.
> Self-driving is not necessary to maintain that valuation, only continued growth of the electric car market in general.
Disagree. The key properties of the car business are high capital costs and high variable costs and not huge margins. The key property of the tech business is low variable cost and often low capital costs (but not always) and high margins.
There's nothing "tech" about building electric cars vs. normal cars and 10 years from now the margins and capital expenses of electric car business will be like the existing ICE car business.
So this is Tesla saying: "Yeah, our financials are starting to look like a normal car company, but we've got this thing that you should keep value us even more like a tech company than you do today."
> 10 years from now the margins and capital expenses of electric car business will be like the existing ICE car business.
Tesla already has margins similar to existing car companies.
> So this is Tesla saying: "Yeah, our financials are starting to look like a normal car company, but we've got this thing that you should keep value us even more like a tech company than you do today."
Yes, they're saying that, but the market is obviously not buying it.
Well, other than actual investors.
Well the middle ground is the actual stock market where Tesla is trading around $265.
How many people and companies are going to freak out if Elon and his team deliver FSD without LIDAR? Say in 2022, not even 2020. My layman logic is that AI is improving exponentially, and we do not know what artificial vision can do in 2 years.
I just spent the weekend racking up several hundred miles in a brand new Model P3D. Phenomenal performance, I am completely sold on the future of EV. I was underwhelmed by autopilot however. Drives like a drunk. The adaptive cruise control part actually made me motion sick. I had high hopes going in, so it was a pretty big let down.
I suspect Tesla is going to find themselves on the other end of a class action lawsuit over all the FSD upgrades they've been selling.
Tesla, on the other hand, thinks this is too fragile to changing environments and works with regular (think google maps level of detail, probably a little better) maps, combined with local semantic information such as signs and lane markings. This also means that they put less pressure on localization, because they don't use the map to detect i.e. speed limit (I'm pretty sure Waymo can detect those signs, it's just that the existence and position of the sign is already known in a global world frame).
They have contracts in place with the owners of cars they have sold preventing them from letting Uber and Lyft use the self driving capability to compete.
Let's hope he is right.