Comment by Animats | Hacker News Reader

Comment by Animats | Hacker News Reader

Parent Full thread

From the article: "Auto-steer had to wait for the freeway, where I toggled it on. Unfortunately, I toggled it right back off. The portion of I-45 near my home is undergoing a multi-year $6 billion rebuild, with a long stretch spanning several exits being destroyed, reconfigured, widened, and otherwise "improved." The result is a multi-mile segment of road that snakes back and forth across what will eventually be both sides of the finished project. For now, there's no consistent surfacing, no real lane markers, and nothing for the lane-keeping system to see."

"Lane keeping was almost excellent. The car kept itself squarely planted in the middle of the lane without ping-ponging between the edges like some earlier systems did. The car would still sometimes "lunge" at exit ramps when lane markings would fall away to the right or left, though the behavior was never enough to make me grab the wheel and disengage."

So Tesla just has lane keeping and smart cruise control, like everybody else. NHTSA Level 2. But they're way ahead in PR.

Google can handle construction, changed lanes, and offramps, city streets, and follows navigation directions. They're NHTSA Level 3.

Handoff from automation to human in emergency situations is still a big problem. Here's such a case. [1] Car with autopilot rear-ended stopped car on freeway. Why? The driver tapped the brake while approaching the stopped car. This disengaged the autopilot, including the emergency braking function, under the assumption the driver was now in charge. Tesla claims the driver should have understood that, as of firmware version 6.2, the auto-braking function was disabled by any manual braking.

[1] http://arstechnica.com/cars/2016/05/another-driver-says-tesl...

Having emergency braking turn off in the event of a manual brake tap is definitely a failure on Tesla's part. Emergency braking should not only not turn off it should brake harder (to the expected traction limit) and then, of course, engage abs if the traction limit is exceeded. Emergency braking should be on unless explicitly disabled (just like abs and traction control are auto on in cars that have them).

> Having emergency braking turn off in the event of a manual brake tap is definitely a failure on Tesla's part.

Yes, but the failure mode is not so easy to see prior to use. It's due to how humans react to the new feature itself, not because it's a bad feature, or a misunderstanding about how users 'normally' react. User's may incorrectly anticipate a behavior (such as in this case), or they may expect the feature to fail so unnecessarily take control (cause of some airline disasters).

This is a general problem of computer human interaction. I'd say it falls under the discipline of UX design. Knowing that breaking is an automatic reaction should be a clue that it will make it a poor mechanism to understand user intent. You then know that you should be looking at other mechanisms to more accurately capture user intent.

- Perhaps the auto-breaking feature should have better defaults such err towards enabling rather than err towards disabling.

- Perhaps the control for toggling auto-breaking should not also be the breaking mechanism itself.

- Perhaps context is now relevant, and user input should be interpreted differently when auto-breaking is on and there is an immanent collision.

I think it's important to pay special attention when making any interface change to a well established user behavior, and acknowledge that new behavior leads to unstable and unreliable input. In these cases put extra effort into finding the path of least cognitive burden when the accuracy of user input is the most critical (i.e. immanent collision) even at the expense of adding cognitive burden elsewhere.

Cognitive burden is a critical consideration in all system design from airplanes to cell phones, but tragically is often overlooked.

> It's due to how humans react to the new feature itself, not because it's a bad feature, or a misunderstanding about how users 'normally' react.

It is not easy but it should be done. Keeping a car steadily between lanes is not easy either, or take self driving cars that is hard. Besides, research into some of this has already been done with aircraft and pilots.

> It's due to how humans react to the new feature itself, not because it's a bad feature, or a misunderstanding about how users 'normally' react.

It is a bad feature. Feature doesn't exist in a void, it exists only because humans are using it. "Human presses the brakes, what to do next?" is a feature with a human in the loop.

> Cognitive burden is a critical consideration in all system design from airplanes to cell phones, but tragically is often overlooked.

Exactly. Moreover "mode confusion" (human not understanding what mode the aircraft autopilots or other systems are in, and then reacting assuming a different mode) is a common cause of crashes.

There were tools and studies specifically geared towards mode confusion:

http://shemesh.larc.nasa.gov/fm/fm-now-mode-confusion.html

Asiana 214 is a recent example of pilot assuming the autothrottle was on, but it wasn't, so plane crashed. It is a bit like here, the user assumed Tesla would slow down some but then stay in the automatic mode.

I believe you mean auto-braking, not auto-breaking, in all the instances you used the term, right? That is, we want the vehicle to brake, but not break.

So you're saying my subliminal messaging was too obvious? ;)

>> Emergency braking should be on unless explicitly disabled

I'm trying to think of a scenario outside of a mechanic's garage that emergency braking would ever be disabled. This should really be mandatory on all cars.

It's surprising that Tesla made that choice.

BMW's anti-collision system is on even during manual driving.[1] BMW will brake even if the driver is pressing the accelerator. So will Volvo. Top Gear compares the various systems as of 2015.[2] Volvo did best; Mercedes didn't stop soon enough and hit the dummy car. VW's city car only does this up to 18MPH.

Maybe Tesla has a false-alarm problem that BMW and Volvo have overcome.

More to the point, this behavior needs to be standardized, or people shifting from one car to another will do the wrong thing. The National Transportation Safety Board is working on this.[3] In a few years, anti-collision may be as standard as anti-skid is now.

[1] http://www.theverge.com/2015/1/5/7494989/bmw-i3-self-parking... [2] https://www.youtube.com/watch?v=PzHM6PVTjXo [3] http://www.ntsb.gov/safety/safety-recs/recletters/H-15-008-0...

I love, love, love the dynamic cruise control on my 2014 Mazda 3 Astina. I stretched my budget for this, our 2nd car, as I was dead set on getting next generation safety features. Got an ex demo manual, couldn't be happier with the overall package.

Even though my car is a 6sp manual, the automatic distance keeping is making the commute much more relaxing. I can't recommend it enough.

Funnily enough the radar system is separate to the AEB, which uses cameras. AEB stands for automatic / autonomous emergency braking.

As I'm someone who likes to know how stuff works, especially in cars for familiarity should a real event occur, I did set out to experience how AEB works. I rigged some large but light cardboard boxes, reversed down the road a bit, accelerated quickly to around 40 km/h, kept the throttle pedal steady as though if to drive through them.

All times, although it felt like surely it wouldn't, the car intervened at the very last second. The first time it drove through them by a metre. The next two times the car stopped before impact. System uses 100% brake force engaging the ABS and also putting on the hazard lights for good measure.

I came out highly impressed. It ought to pick up cars better, but I'm not about to actively try that.

In some urban environments it is necessary to nudge very slowly through pedestrian crossings to thread through gaps; if the car brakes at the sight of any perceived obstacle and waits for a totally clear right of way, there are certain crosswalks and right turns in Chicago you will not pass for several hours.

Well if you're going under 5mph then it's not really 'emergency braking' any more.

Good point. For those situations, there should be a separate override button to disable the feature, just like there often is one for traction control (on normally, button lights up when disabled). Or for the passenger airbag back before weight sensors replaced a manual override.

But otherwise, even when in manual driving mode, the emergency brake feature should be active and watching for impending collisions.

Just make the emergency brakes kick in only above 5km/h?

It is, but if when it starts braking you press the brake pedal, it turns off under the assumption that you want to manage braking.

That's why it should be a separate button, so that it never assumes you want to manage braking unless you specifically turn the system off. Just pressing the brake pedal doesn't mean you want automatic braking to go away.

> I'm trying to think of a scenario outside of a mechanic's garage that emergency braking would ever be disabled. This should really be mandatory on all cars.

So there's an accident, and I swerve to avoid. I think I'm going to make it but the car isn't certain so it brakes, and it brakes really fucking hard. At this point who is responsible for what happens? Breaking like that fundamentally changes the behaviour of the car. If I had just braked, maybe I stop in time maybe I don't - I chose another option and it is completely valid.

The very idea of that kind of behaviour terrifies me. I'll take a 100% automated car with pleasure, but I have no interest at all in this hybrid control concept. I would turn that shit off on delivery.

> So there's an accident, and I swerve to avoid. I think I'm going to make it but the car isn't certain so it brakes, and it brakes really fucking hard. At this point who is responsible for what happens?

I'm having problems understanding your scenario. How exactly does the car breaking prevent you from swerving? The brakes should only stop your car mid-swerve, and you wouldn't collide. The only scenario when speed is an advantage when swerving is when you're trying to needle into an opening in the next lane and you need to match the speed of a 3rd car

When swerving you should not apply the brake. It will cause you to lose control in the swerve.

Not with a decent ABS system, and all the cars that have automatic braking have pretty good ABS.

Braking uses all the traction between your tires and the road to slow down your car, and you will be unable to turn. In the case of ABS, you will turn at a slower rate than if you hadn't been braking.

Which promptly trains all drivers into a "meh" attitude toward braking, leaving it to the computer, and then surprised into paralysis (and subsequent crash) when it doesn't work.

law of unintended consequences. I can even imagine a net increase in accidents until the technology is mature enough to actually take over full responsibility. So far all of the assistive technologies were straightforward to understand and control - by that I mean ABS, ESP, velocity keeping, and autobraking. You were never tempted to think that the car will drive for you with these. But the moment you add lane keeping and take over the steering wheel, there's no immediate obligation to pay attention anymore. It's deep in the uncanny valley and I'll have none of it until we have fully autonomous cars.

Another user linked a Fifth Gear video testing the auto-braking systems in Volvo/Mercedes/VW cars. In it, they say the auto-braking systems only engage by braking hard at the last second specifically so you don't get blasé about it as if it were a smooth controlled braking. It's not a mode you want to use because it really feels like you're going to crash. https://www.youtube.com/watch?v=PzHM6PVTjXo

that's interesting, I can see the reasoning behind, but what happens on a slippery road?

That sounds like exactly what happened in the crash scenario. The driver tapped the break, said "meh, the car will handle it" and then plowed into the car in front of them.

I'm pretty leery of these systems at the moment due to people using them like this. That's just not what they are ready for at the moment and people's unrealistic expectations are leading them down dangerous paths.

Is this not basically what Autopilot is doing for driving in general?

Humans can be trained into not having surprise paralysis, for instance, through a simulation. It can be part of the requirements for driving permits.

fog / rain / low visibility (potentially fooling the sensor into thinking there's an object in front of you), cars behind you?

That raises an interesting question: It is 2016, do we ( or do we not ) currently have economically feasible technology to place into cars that can reliably and accurately determine when you are about to hit a solid object or not?

Define "Solid object".

In the lab, opaque textured objects in sufficient lighting are pretty easy to localize by a variety of methods. Reflective objects are quite a lot harder, but there are methods that work. Reflective objects through distortion like rain/fog... harder, but often doable. In low light? Depends.

The weird thing with environment modelling in computer vision is that there are a lot of overlapping methods, but sensor fusion of these methods is rarely attempted. Most robots end up with a pair of laser rangefinders or a pair of cameras or something equally simple to program. Creating a high-reliability automated vehicle that can bypass all the edge cases where these things fail is another thing entirely. You want overlapping, inter-reinforcing models of the world, with algorithms that not only identify items in a well-calibrated system, but calibrate the system in real time based on items identified.

My lab was really excited about replacing stereoscopic computer vision with monocular computer vision for 3D modelling, but the opposite direction is where you want to go if you care about as many 9's of reliability as possible and hardware is cheap. You want a few dozen cameras with fixed orientation, you want parallax laser rangefinders in a variety of bands, you want time of travel LIDAR units, you want a {GPS + 3DoF gyroscope + 3DoF magnetometer + 3 DoF accelerometer + barometer + thermometer}†, you want ultrasonic and radar rangefinders; You want everything you can get. You want enough lenses that if one gets obscured by water, hey, there are others that will pick up the slack, and do it without bothering the driver.

You want to shrink the 'no coverage of object in environment' surface as small as possible.

Nobody seems to respect this on the academic software side. Novelty, sure, but people don't seem to earn grants publishing extra 9's using a mashup of a bunch of pricy sensors. 'Less is more' is not how you get the data needed to make things reliable, it's how you get best-case detection rather than worst-case awareness.

†This combination is now found in quadrotor IMUs. Each individual sensor is inaccurate enough that it would be useless for automation, but combine them to check each other's deficits and you get a remarkably elegant system.

Where the solid object is a car or truck, yes. Phased array radar is good enough for that, and there are lots of deployed units on the road. Telephone poles, deer, and pedestrians - maybe.

The canonical example would be a plastic bag on the freeway.

The point is any anti-collision system should be smart enough to tell the difference between a car and a bag. It should NEVER let you plow into another car just because you tapped the brake. It shouldn't even matter if autopilot is on or off. Only if this was a child or animal, could I see the bag argument.

Also, it was a UI failure not notifying the driver clearly that auto-pilot has disengaged. Even the primitive cruise control in my car has a big green light, that turns off when you tap the break. The more autonomous a car is, the more important it is to clearly notify the driver of control handoff.

Well of course, but there are always edge cases. Large rock, or a football? A person, vs. a cardboard cutout of a person? An empty plastic bag, vs. heavy bag with clearly something in it? A human can instantly tell the difference between all three, but for a computer to do it you would need some extremely good image recognition(in the case of a loaded bag the problem becomes near impossible, it's hard enough to just recognize something as a bag, being able to tell if it's weighted requires spatial awareness that no one proposed how to solve yet). In the meantime out best of the best image recognition software can't tell a zebra and a sofa in a zebra print apart. So for now, while we wait for the algorithms to improve, there should always be an easy (with an emphasis on easy) way to disable automatic breaking.

Granted, ABS invalidates the following argument a little, but not entirely. The traditional wisdom when faced with bad road conditions and an impending accident is that maintaining control through steering is much preferable to braking, which lowers the speed with which you can manoeuvre out of the situation.

It all depends so much on various variables so I find it hard to categorically say "it should always be on."

You are going 80mph on a motorway, and a large shopping bag blows into your way - having the car do an emergency stop in that situation would be extremely dangerous, so a gentle tap on the brake to tell that car "yes, I am in control of the situation, there is an obstacle in front but I don't mind hitting it" is the only solution.

Signalling the car "yes, I'm in control" is a good and only solution, however, tapping the brakes is (a) definitely not the only way to signal that; (b) not a intuitive way to do that if you really want to make this signal but aren't sure how; (c) not a safe interpretation, as it's likely that people will tap the brake with other intent than the signal "it's ok, I got this".

An appropriate way to signal it would be distinct from any emergency controls and also show a highly visible signal that you've turned the system off and are in a different "mode"; even a simple separate dashboard switch with a light would be better for it than the brake pedal.

I can think of two top of mind.

A deer crossing the road in icy conditions. Its better to hit the deer than lose control.

a large tumble weed blowing across the road with someone tail gating you.

I would not want to write that if statement.

Lane Keeping on Tesla is miles ahead of Lane Keeping on other cars. It centers in the lane, slows down and anticipates curves, etc. It is really a game changer vs. traditional "lane keeping"

I've never driven a Tesla to compare, but the 2017 Audi A4 does lane keeping much better than I expected and would say the A4 is comparable based on what others have said about Tesla.

I think Mercedes has a pretty comparable system as well, but can't speak from my own driving experience.

It didn't include Audi, but FWIW the only head to head comparison I've read put Tesla far ahead of Mercedes, Infiniti and BMW - http://www.caranddriver.com/features/semi-autonomous-cars-co...

I'm still wondering what an automatic car(or merely one that has an automatic lane-keeping system) is going to do on a road like this:

http://gfx.dlastudenta.pl/photo_new/d90/b27/097/537/102764

Or like this:

http://img.sadistic.pl/pics/a87df587b218.jpg

Not everyone lives in countries with amazing road infrastructure, the pictures above were taken in a country that is a member of the EU,not some far Russian province.

Bring the car to a halt, and ask the user for help?

I've riden in a Tesla and I can assure you it does not anticipate or correctly react to all curves. For example, the 2016 Tesla S totally over reacted to this slight jog in 101, terrifying all of us, and forced the driver to take over. IMHO, not yet game changer.

https://www.google.com/maps/place/US-101,+Mountain+View,+CA+...

> Google can handle construction, changed lanes, and offramps, city streets, and follows navigation directions.

Oh that's great news! Now where can I buy a Google car again?

There's a big difference between a general production-ready car and what's basically an internal-use-only car.

Hmmmm highway conditions vs 25Mph, the top speed of Google's self driving cars. The Tesla Autopilot is a lot more impressive as it goes a lot faster than 25Mph.

It's also really good at 25Mph - when the roads are not busy in my area I take it out and drive slower and on curvy roads that might be normally 35-40 I slow down autopilot to say 25 and it's really good. At speed there are still a few turns it misses. A year ago at speed it couldn't do even half of what it can do at speed today. So it's also improving with every update

Google went for a 25 mph top speed for regulatory reasons (the car is basically classified as a golf cart that way), not for technological reasons.

I would be absolutely fucking terrified to engage autopilot at all in Mountain View, let alone at 25mph.

It's a great feature, but it's a great feature for boring driving.

> it's a great feature for boring driving

I'd much rather have a "boring" driver on the road than deal with aggressive drivers.

Agreed. My point was that it doesn't deal well with "exciting" driving. Given a long stretch of mostly empty interstate highway Autopilot is great (and keeps it suitably boring/low stress). It does miserably in suburban areas (let alone urban areas).

Reply on news.ycombinator.com