It seems like they should have a million hard test cases that must pass in simulation before releasing a new model. The simulations should be harder and more extreme than anything encountered in real life.
I think the real problem is obvious. They're trying to rush the work because Elon said so.
The car is properly recognizing traffic lights pretty darned well, considering the circumstance. It looks like it has a built in understanding that traffic lights are "always" stationary - hence, assigning them static locations on the 3D map - but it keeps having to update the model because the lights are actually moving.
This seems like a very non-obvious edge case that I wouldn't expect an ML team to even consider as a possibility. Now they need to program into the ML model an understanding that traffic lights are typically stationary. Which seems even more difficult to me, from a technical perspective - you don't want false negatives...
The car isn't braking or making any strange maneuvers from what I can tell. I'm actually impressed that it's handling it this well.
When you don't know what to do - do nothing. What if it was a traffic light on roller skates? Or a kid, dressed as a traffic light, on roller skates?
Collision detection systems (radar) are accurately not detecting an impending collision because the lights are not actually on a collision course with the vehicle.
Object recognition systems (computer vision) are working very well, because they recognize the lights and are updating the 3D map accordingly, but the traffic light 3D model is not designed to be a moving object - unlike vehicles, which frequently move. Which is why we see the car "passing through" them.
What we are likely seeing is simply a weird edge case in the output for the user-interface. I'd imagine if an object was actually flying at the car and the car could see it, it would brake accordingly.
Also, the map is two-dimensional. The car frequently drives underneath traffic lights that I'm sure also appear "on top of" the car in normal cases.
Object recognition and collision detection, from what I understand, are two very different systems.
But seriously, I’m inclined to be charitable here and assume that this is merely a quirk of the UI display. There’s no evidence that the autopilot did anything unsafe (apparently it wasn’t even engaged?), and until I see evidence of that I’m willing to withhold judgment. (I have seen evidence of other situations where Tesla autopilot did unsafe things and I’m in no way apologetic about those situations.)
In this case, obviously it's not a safety concern because the object is being hauled on the back of a truck.
They don't work there anymore.
From their experience, I know one thing: I will never work for Elon Musk. He may be a great visionary and salesman, but he's a horrible manager.
It was stunning to see the complete disconnect between Musk's grand declarations and what the organization was actually setup to deliver.
Frankly, it just gives me more respect for Tim Cook, who as COO at Apple made his company able to turnaround and deliver HW in record time.
Edit: in retrospect I wonder if Musk's grand public declarations were actually a way to control and pressure his own organization. Remember, Musk didn't actually found Tesla, he rescued it from bankruptcy after the Roadster didn't return as much as needed, so he inherited an existing structure.
I know it's cool to hate Elon.
If you want evidence from a reasonably neutral observer, take Sandy Munro (himself an engineer who has worked on everything from cars to aeroplanes). He recently interviewed Elon, ostensibly about Tesla but the interview was in a meeting room at SpaceX. After the interview he was invited to a two hour design review meeting and was "blown away" at Elon's depth of involvement.
https://youtu.be/S1nc_chrNQk?t=370 (6:10 to 8:45)
Sources: https://www.reddit.com/r/SpaceXLounge/comments/k1e0ta/eviden...
Yes. The human brain and visual systems aren't nearly so trivial to replicate as a lot of people in the tech industry seem to think.
Tesla is just one of many case studies in the paired tech industry arrogance seen so frequently:
- "A human is just a couple really crappy cameras and a neural network, we know how to do better cameras and neural networks, how hard can it be?"
- "We can do anything we dream with 99.995% reliability in the synthetic, computer-based world of the internet because we know code. Therefore, we can do anything we want in the physical reality with code!"
Both are far from evident in practice, but the belief in them continues, despite it being increasingly obvious to everyone else that neither one is true.
Human vision and world processing is quite impressive - and, as pointed out elsewhere in this thread, a two or three year old would have no trouble working out that the obstacles were some things on a truck. I've got a nearly three year old, and I guarantee he wouldn't confuse those for stoplights in the slightest. I also wouldn't let him out on the road, though he does well enough with a little Power Wheels type toy. But there is far more going on in the visual processing system than we even understand yet, much less have the slightest clue how to replicate.
And while code may be fine on the internet (where you can retry failed API calls and things mostly make sense), the quote about how fiction is constrained by what's believable and reality sees no such restrictions is very true. Out on the roads, all sorts of absolutely insane things can and do happen on a regular basis - and you can't predict or plan for all of them. But the car has to handle them or it crashes.
As a random example, a year or two ago, I was behind a car that had poorly strapped a chunk of plywood to their roofrack with a good chunk hanging forward, and the front end of it was starting to oscillate awfully hard. I had a good clue that it was going to come apart sometime in the very near future, so backed off from a normal following distance to quite a way back. Sure enough, half a mile later, it failed, went flying through the air, slammed into the road a good distance behind the car, and tumbled a bit. Had I been using a normal in town following distance, it would have either hit me or tumbled into me, but using a human visual system, it was obvious that my existing following distance stood a good chance of being a bad idea.
Stuff like this happens on roads constantly. Meanwhile, state of the art self driving can't tell the difference between stoplights and some poles on a truck. You'll excuse me if I don't think the problem is anywhere remotely close to solved for a general case Level 4 purpose.
Aritical neural networks are pretty good at object recognition, among hundreds of other things, and even better than humans at some of them. They are, however, generally pretty bad at abstract reasoning, critical thinking, 'understanding' concepts in-depth, and so on, and I think that's a more constructive way to phrase the problem we see in this video.
When a problem is fully redicible to a simple vision problem, modern neural networks are a great choice, but being a good driver involves much more than just the visual cortex.
One way to think of this is that that the footage is implicitly labelled: we have the benefit of hindsight: we know what the state/location of the vehicle was going into the future. That benefit of hindsight also can serve as implicit labels by knowledge that the vehicle did not crash or collide with something immediately after the footage.
I don't understand why this is hard for Tesla engineers -- I was doing this kind of thing in grad school a decade+ ago and it worked fine. I've seen it in other demos where object classifications rapidly cycle between person, bike, car, etc. Are they not filtering anything? Is this a symptom of "AI-ing all the things"? Because we did it with bog standard computer vision techniques back then and never got behaviors like this.
Move fast and break things [like tests]