Comment by danblick | Hacker News Reader

Comment by danblick | Hacker News Reader

Parent Full thread

Miles of data is not the best metric. If you've ever tried to build a machine learning system you'd recognize some of the issues here. First off, where are the labels? How are you to interpret all that data?

Say Tesla has recorded video of some dude driving down his street in LA and their system detects a car door opening in his data stream. How can Tesla find out whether or not there actually was a car door opening? Human labelers (for billions of miles of data)? Multiple instruments?

If I were designing a system, I'd want a car with extra instruments on it that would help me validate what the primary instruments detected. The idea of just recording a bunch of live traffic seems pretty lame. (You have no way to validate any of the conclusions you draw from the data and hence no way to improve your ability to draw conclusions from the data.)

Did you see the article about Waymos test environment at Castle? That, along with a boatload of simulation, is exactly the kind of testing you need to do to develop this kind of a system.

Has Tesla ever demo'ed their simulation environment or other tests?

Not sure why you think simulation environment is more valuable than real world data. Waymo doesn't even have as much data in simulated environment as Tesla does in real world data, which is hilarious.

In a simulation environment you know ground truth. (Where objects actually were vs. what you saw.) You can play the same scenario from slightly different angles. Or different conditions (rain, snow, low light, bright light). You can look at counterfactuals (what would I have seen if I had done X). You can test the behavior of your system without actually setting a computer loose in the real world.

From "real world" recorded data you have none of these things.

This is the difference between observational and experimental data and it's a huge difference.

Agreed with everything you are saying -but my understanding is that the Waymo simulation system doesn't simulate Input, it simulates the objects. The whole idea being that responding to objects is a separate problem from perceiving the in the first place.

Waymo has claimed they do 8 million miles a day, and it's in simulation that they've made the lion's share of their progress the past few years. In simulation you can model thousands of variations of any particular corner case, the AI doesn't know the difference between real and virtual environments.

Dedicated test vehicles are collecting many terabytes a day. Google mentioned years ago that they're gathering over a gig per second of data from their test vehicles, and it's likely more than that now. Tesla is gathering a gig or 2 of data at a time from it's hw2 equipped cars here and there. I'm optimistic that Tesla can, by the end of the year, get to about where Google was at in 2011.

Reply on news.ycombinator.com