Hacker News Reader

Comment by dmitrygr | Hacker News Reader

Further proving that "self driving" is approximately as hard as AGI, and is nowhere near as "close" as everyone thinks to think it is. A human would have context and common sense and thus know that a residential street will simply not ever have an 85 mph speed limit.

And yes, you could special case this case in the code, but there are hundreds of cases where "common sense" is used in driving, and you will never teach your NNs all of it.

Prediction: no actual "self driving" (to a point where driver can legally be asleep) on public streets till > 10 years from now

View on HN

Is no one pointing out that this is the older Tesla Model S that doesn't even use Tesla's autopilot? It's MobilEye. MobilEye patented reading speed limit signs, so Tesla's current autopilot doesn't even do this.

People saying "see, THIS PROVES self-driving is as hard as AGI" without even applying any critical thinking is just confirmation bias.

You can tell the Model S is old because it has the pre-April-2016 bubble nose. I believe all vehicles with the bubble nose have the old MobilEye system at best (which was never intended for self-driving).

Him: self driving is more than a decade away. You: No, because this attack is for an old vehicle (implying that newer vehicles are able to evade it). You see, the new cars don't have this feature at all!

Cool, so the new cars have even less features than before. And that justifies that self driving is near?

Not really, he is saying that argument is not strong enough to say anything about immediacy of self driving.

> I believe all vehicles with the bubble nose have the old MobilEye system at best (which was never intended for self-driving).

If the MobilEye system is not trustworthy then why is the vehicle accelerating based on the data it receives from it? Why doesn't Tesla update the software to stop doing that?

I don't see why you need special cases in the code. Sat nav units have had speed limits built in for years. Why not just look it up in a database? (the car knows where it is after all, and requiring a GPS fix could be mandatory - most drones won't arm without it, for example)

Because those databases are out of date approximately 100% of the time...

Road construction, for example is a reason why

However if the sat nav gives you a prior that says 35 mph, and the road sign says 85 mph, do you trust it? Especially if the adjacent roads are also low speed, and it's clearly not a highway etc. This is perhaps a bad example as there are plenty of roads in the UK that transition from national to 30 in rural areas (and plenty of hamlets that you can bomb through).

Roadworks are an obvious situation where you drive at a _lower_ speed. This would be a case where you could use computer vision to identity an obstruction, identify roadworks and then adjust speed.

Those databases shouldn't be out of date. Most highway agencies (UK for sure) publish planned works and improvements weeks in advance. Google and Apple both show road closures for example. I belive speed changes are similar.

Google maps keep saying 101N in San carlos is 65mph. It's been 55mph due to construction for 5 months...

This is literally Google's back yard

65 in a 55 is much less dangerous than 85 in a 35. It wouldn’t be common that road conditions would permit very high speeds while at the same time having an artificially lowered speed limit (e.g construction).

This shouldn’t be a problem, in general, with a couple extra sanity checks.

I would expect the correct solution to more commonly have the opposite problem: too slow in a faster zone.

Even so, what's the worst-case scenario in biasing towards lower speed and assisting your vision system with a knowledge base?

You perform the task of vision 100% of the time driving. From a purely probabilistic standpoint, even assuming highly accurate ML, the relatively infrequent database updates still make for a sensible prior.

This is somewhat adjacent to what Gary Marcus has been arguing, and I think it makes a lot of sense; there seems to be no compelling reason to rely exclusively on primitives (i.e. vision) when good priors are easily accessible.

Out of date is probably not that big a deal. If a speed limit is lowered from 40 to 35, or even 65 to 55, and the database doesn't get updated, it's unlikely to create a dangerous situation if a car doesn't know about it.

But a car thinking a 35mph zone is an 85mph zone... that could be a very big deal indeed.

Regardless, if self-driving cars (or at least "computer assisted driving" cars) were to be widely deployed enough and shown to depend on these sorts of databases, I'd expect it to become a legal requirement to make sure these databases are kept up to date by city/county/state agencies, and that car systems get those updates. Otherwise, expensive liability ensues.

It gets complicated fast. Currently if there is some sort of emergency or requirement to notify road users of a change to driving conditions, a person can just put a sign up. It mostly just works. If road workers are also required to update a database that needs to be read by a car, that’s a lot of infrastructure that needs to be working. You also need to have the humans and machines reading the signs as the same at every time point, as “road closed” or “one way” needs to be correctly understood by everyone at the same time. In an emergency that may involve bad weather or some other sort of disaster, a remote database is not going to work very well. The car needs to read the actual sign properly.

Waze knows the speed limit and warns you when you exceed it.

Except when it "knows" wrong, or the limit is conditional (days, hours, vehicle classes), etc.

How much can you trust that data? My car tell me what the speed limit is by using a database and reading signs. On my commute to work there is a part where it tell me the speed limit is 5km/h when it actually is 80km/h and another place where it for a short bit shows the limit as 80km/h where the correct is 30km/h

In The Netherlansds (and probably other places), many roads have a different maximum speed between 07:00-19:00 and 19:00-07:00.

Both Netherlands and Germany have these time based speeds on a lot of highways where the special case is in text underneath the sign.

BMW handles it if the navigation database is up to date. At least on the A2 in the Netherlands, which has the 06:00 to 19:00 changes between 100 and 130. It also understands temporary changes for roadworks, but gets it wrong when a roadwork/temporary sign has an exception like "when road is wet"

That tells me the camera only understands the main sign and then the navigation database tells it about exceptions / special cases. Nice as a small reminder on your screen, but I wouldn't trust it to self drive based on that data.

Neither is ideal. But you should use what information you have. There is still no excuse for making a decision like 85 in a 35. Worst case this should be a disengage or the car could defer to you - eg visual warning on the dash "speed limit unclear, override?"

In the UK smart motorways can vary from 30mph to 70mph according to traffic. This can change within minutes and obviously cannot be in databases. So this is a good case for sign recognition. In theory though, there is some network that's updating the speed limits and there's no technical reason why that couldn't be a public information service (maybe it already is).

Then you have the sibling comments - speed limits which are time based. I would imagine this is country specific and is, mostly, a matter of public record. But would you trust CV to recognise text on a sign with specific hours? I don't know how complex the signage is, but trying to use ML for this seems absurdly wasteful of resources.

Finally you have other information - how fast are the cars around you travelling? Have you detected an obstruction on the road (eg cones)? Is the weather inclement and therefore you should drive more slowly, etc.

I would imaging that if self driving cars become ubiquitous there will eventually be a system for real time speed limit determination, whether that's some kind of wireless beacon at regular intervals or an online database that's kept up to date I don't know.

How is the accuracy of sign recognition on the LED signs stating the current speed limit? My experience is that it is not so good, maybe other cars are better at reading LED signs.

For the car to react according to how other cars drive, you need to have other cars on the road at the same time. I have a narrow gravel road where the limit is 30km/h but safe driving is between 10 and 20, but my car tell the limit is 50km/h. It can be weeks without me seeing other cars on that road at the same time as me. Some mornings during the winter I can drive because I know where the road is, not because all of it is visible. Good luck having an autopilot doing that

Well, quite. But again this is all supplementary information. The car should have a basic idea of the speed limit from a database and it should adjust as appropriate.

Similarly the car should be able to identify road surface conditions and also how much power it's needing to put in to achieve a set speed. If it's identified that traction is being lost over gravel, then it should reduce power.

Equally the autopilot should consider visibility. There are plenty of roads that are national speed limit and you'd be insane to drive at 60 on them due to blind or extremely tight corners.

But with LEDs this seems like massively over-engineering the problem. Why use computer vision when you can potentially broker a deal with the highways agency to get live speed limits on a particular stretch of highway?

Tesla already looks it up in a database.

Current Teslas CANNOT have this error simply because MobilEye patented reading speed limit signs and Tesla no longer uses the MobilEye system and haven't since mid-2016.

side remark : how can you patent a "function" ? Wouldn't mobileeye have to patent a certain way of reading speed limit ?

It is called a process patent.

Thanks for the enlightenment!

The ability to accurately map surroundings is a fundamental prerequisite to AGI but it's pretty much the whole meal for self driving.

From there the car needs to segment its interpreted reality and run path finding algorithms.

AGI needs to take that reality and seemingly consciously understand the many different ways that it can interact with it.

I'm oversimplifying the post mapping implementation of self driving - but not so as to remove an insurmountable problem.

Tesla's biggest fuck ups (such as driving under a truck) seem to me to be the consequences of initial mistakes in building a map of reality, rather than in what's being done with that map.

For self driving, as far as I can tell, the map of reality is the hard part.

For AGI, it gets you to the hard part.

I think "> 10" is even still pretty optimistic. I'd be surprised and impressed to see driver-asleep self-driving in the next 25 years, unless we see some serious unexpected breakthroughs in computing.

It's probably closer than you think.

The reason we don't already have self driving cars is that there hasn't been enough computing power in a small package to make the sort of decision logic needed practical until recently.

As long as the prerequisite for self driving has been to emulate human's driving skills, there just hasn't been the computing power available. Now that there is, progress is being made, but really it's just a transitional period.

What's eventually going to happen is that roads everywhere will get machine assists - basically, small beacons running on low power that provide easy machine readable data on everything from precise road location to speed limits to local road conditions. They'll be small, low powered radio transmitters or even NFC devices near the road that the car can just download data from.

There's always been a chicken and egg problem with self driving vehicles, where no one wanted to invest in machine assists for all roads because no one wanted to put that much money and effort into something that no one needed (no one had self driving cars) but no one wanted to make self driving cars a product that wouldn't be successful without machine assists as well as a way to drive without them as a backup.

Now that self driving cars are out there and everyone is getting comfortable with the concept, machine assists will be coming, and they're going to have benefits to ordinary drivers, too.

If you want an example of why this will work, look at aircraft. Most jet liners today are perfectly capable of landing themselves in almost any weather because of the electronic navigation system they have built in to most airports. The planes aren't using machine vision and AI code, they're landing on a piece of ground defined by numbers in the navigation system which is precisely located by GPS coordinates.

The same will work for cars, it just hasn't been built out yet.

Did you ever try to play GeoGuessr game? I'm not living in USA, I'm living in country which is infamous for its notoriously bad roads (you could say that it is 3rd world country and I will not blame you), so I will not use my homeland as example. But when I'm playing GeoGuessr in USA I often found "myself" on small road with not-so-good tarmac, with wore-out road markings, without any light posts… Google car was here (or there will be no street view), but still. Are you sure "roads everywhere will get machine assists" soon? Who will pay for that, if now nobody pays to fix tarmac and road markings, not to say about lights installation, which will help EVERYBODY RIGHT NOW?

Airports are completely irrelevant example. There are ~2500 IATA recognized airports out of ~50K "airfields" total, and only ~1000 of these 2500 are equipped with ILS. It is not comparable with millions of kilometers of roads around the planet, even if we will count only "1st world countries".

And a lesser number of airports (specific runways of those) still support CAT3 autoland...

It only failed on older gen1 MobilEye hardware. So this particular vision problem has been solved for a long time.

Not only that, but current Teslas don't (and can't, due to MobilEye's patents) rely on vision for speed limits.

Do the NNs actually need to learn this? I’ve always assumed that NNs primarily being used to do computer vision, but that all the driving decisions would essentially be a fancy rule system.

Is this not how most self-driving works?

When the implications of Adversarial AI clicked, I knew the autonomous vehicle thrust as we were lead to believe it was over. No way, no how.

Your predictions might well be true, but this hack is not any evidence for it.

So you can attack an AI driver with targeted sign changes which wouldn't work with human drivers. I don't think you'll need experimental evidence to agree that you could also attack human drivers with targeted sign changes which wouldn't work with a particular AI driver. Relevant xkcd: https://m.xkcd.com/1958/.

An AI driver will remain dumber than a human until we're very close to AGI, if not forever. But it never gets distracted, and always tries its best to follow the rules. This may well be enough to compensate for its low intelligence - and beyond. The worst human drivers ain't always the lowest IQ ones.

> Prediction: no actual "self driving" (to a point where driver can legally be asleep) on public streets till > 10 years from now

We already have "driver-asleep self driving" on public streets: https://www.theverge.com/2019/12/9/21000085/waymo-fully-driv...

Reply on news.ycombinator.com