Hacker News Reader

Comment by tunesmith | Hacker News Reader

Every time this article comes up on hacker news there are a few comments that show up talking about how no one was able to prove that the bugs definitely caused the crashes, and thus the criticisms of Toyota are overblown. I'm left wondering how on earth to reconcile those viewpoints with general software engineering (which is a huge part of what this community is about), where we regularly see articles about new concurrency frameworks, fault-tolerant programming, formal methods to prove correctness of code, etc. If it's important enough for Amazon to use TLA+ to prove correctness of DyanmoDB then shouldn't it be considered outrageous that Toyota doesn't do the same thing for something that is potentially an accelerating death machine?

View on HN

The HN community at large isn't experienced with software engineering. It seems like there are a handful of commenters who are, but they are a very small minority.

Personally, I'm surprised that the majority of the HN community overlooks or ignores the reports from NASA and Exponent (Toyota's outside expert witness) in favor of the testimony from Barr, the witness for the plaintiffs in the Oklahoma case. (I'm honestly amazed that Barr was allowed as an expert witness) Barr did show that the code "smelled" and that it was possible to inject specific faults to induce an uncommanded acceleration, but he did not show that his proposed failure mode occurred nor that it was probable to occur. In fact, part of his failure mode was that it left no record of a DTC. It's the perfect kind of failure mode for a sympathetic plaintiff to wield against an arrogant defendant with deep pockets: possible that it occurred, possible to demonstrate, understandable for a jury (who hasn't had a BSOD?), and impossible to disprove.

You bring up the practice of software engineering, but I think you need to place it in context of how software engineering was practiced around the time that the electronic throttle system was developed. MISRA, which HN readers are becoming aware of, didn't exist. Proof assistants, which today remain challenging to use and integrate into an SLDC, were even more arcane. In fact, your example of TLA+ wouldn't fly even today since TLA+ doesn't do code generation, and that would be a necessary component of the verification process of a regulated SLDC. I don't think TLA+ existed at the time either. Things certainly have changed!

> You bring up the practice of software engineering, but I think you need to place it in context of how software engineering was practiced around the time that the electronic throttle system was developed. MISRA, which HN readers are becoming aware of, didn't exist.

DO-178B was created in 1989, Ada was created in the 70's, standards for mission and safety critical systems have been around since the 70's, I was reading about Ada and safety critical programming as a kid in the 80's (I was a weird kid).

http://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_s... has a very good overview.

Whether you accept that multiple independent experts in safety critical programming are right or not is up to you however the stuff that came out of the trial painted a very clear picture to me of just how bad this software is.

A cynical person might consider that if Toyota is this poor how many other manufacturers are also as bad.

EDIT: I'm not saying that the UA was caused by this, without the source code and some proof I can't say that however the general setting is pretty terrifying.

> DO-178B was created in 1989, Ada was created in the 70's, standards for mission and safety critical systems have been around since the 70's, I was reading about Ada and safety critical programming as a kid in the 80's (I was a weird kid).

I'll hazard a guess that the HN readership consists of many formerly weird kids. :)

DO-178B was around then, but I'm not aware of any automotive groups using it then or now—I'm not in the automotive sector, so I could easily be mistaken. I would argue that IEC 61508 would have been the best model to follow in lieu of something more specific (especially since IEC 26262 is an adaptation of it), but I don't think even 61508 existed at the time outside of a draft. Ada was around, but it was a different beast than it is now and the tooling wasn't very good at the time (my opinion). I can't think of many that were using Ada outside of the government mandate.

For general consumer products (i.e., outside of aviation and military), I'm having trouble thinking of industry standards for mission- and safety-critical electronic systems that existed at the time that Toyota's electronic throttle system was developed. Maybe SAE had something at the time? A cursory Google search didn't pull up anything.

> Whether you accept that multiple independent experts in safety critical programming are right or not is up to you however the stuff that came out of the trial painted a very clear picture to me of just how bad this software is.

To clarify, I'm not saying that the software isn't poorly written by modern standards, but I am questioning that it was uniquely poor relative to the rest of the industry at the time it was developed. That said, Toyota was/is arrogant, and ignoring their own processes and requirements at the time is unquestionably the wrong thing to do. I remain skeptical that the software component was responsible for the unintended acceleration issues.

> A cynical person might consider that if Toyota is this poor how many other manufacturers are also as bad.

Isn't the null hypothesis that they were all equally bad?

I think the relevant standard for comparison, from a safety point of view, is with mechanical throttle linkages that were used before throttle-by-wire. And it's pretty obvious that the mechanical linkages are less reliable: they involve a long cable snaking around the engine bay. A sticky throttle used to be a common complaint. With these systems, it's not. And of course, it's the multi-channel redundant brakes that are the critical backup safety system here.

BTW, I read through Barr's attempted reengineering of Toyota's ECU (his slides are linked in the comments). It's just making me angry. After going on and on about how bad Toyota's code is (spaghetti, blah blah blah), he starts presenting his failure modes: suppose that a random hardware memory error flips some bits in the CPU's task table. Then the task monitoring the pedal angle is going to die. Now to get actual unintended acceleration as described (when the driver is pressing the brake), you also have to suppose that the throttle position variable is corrupted at the same time. Other than the general unlikelihood of this, consider: suppose that Toyota's code did not contain any "spaghetti" or global variables. Suppose in fact that it was beautiful enough to make angels weep tears of joy. Would that make the slightest fucking difference, pardon my Japanese, when you start flipping bits in the task table? Of course fucking not.

His complaint amounts to amateur backseat engineering: you protected variables A and B from corruption by having multiple copies, so why not C? Your watchdog will restart tasks X and Y when they die, so why not Z? And so on. Which is an OK suggestion for the future, but how are they liable for something for not making an already extremely safe system slightly safer, when it's much safer than previous systems, and has a fantastically reliable backup?

Among other things, what I found interesting was that Barr was giving expert testimony on engineering and "engineering certainty", when as far as I can tell he doesn't have a PE license (if he does, he's the first engineer I've ever heard of that doesn't conspicuously advertise it). I was under the impression that wasn't permitted in any state.

edit: Toyota did plenty wrong. Chiefly, I'd say that ignoring their own documented processes should be at the top. Having a system utilization of >70% would be another, as is using recursion. All are things that, at the time, were no-nos. I don't however think that one can fairly argue that it's practical or sensible to implement emerging standards during the multi-year car development process, which many seem to be arguing. I also remain skeptical that the unintended acceleration events are software related. It seems that for one to buy the bit flip argument from the trial, one also has to assume that the drivers depleted the service break vacuum, and that combination doesn't seem probable to me.

Sure, they should improve or clean up the code on general principles. However, there are 2 issues here:

1) A lot of these articles imply that some expert has demonstrated how a coding error will cause unintended acceleration. Then when you look at the actual source, it becomes clear that the "demonstration" involves changing the internal state of the controller in all sorts of arbitrary ways, and sometimes rewiring its sensors in an invalid way as well. In other words, this is not a recipe along the lines of "blip the throttle while changing from N to D, press the brake within 0.2 seconds, the throttle will now be wide open". The fact that no such recipe has been found, despite many millions of dollars spent on expert analysis, suggests that it doesn't exist in the wild. Implying that this code is killing people somewhere out there is misleading.

2) Yes, it has been overblown. We know that throttles can stick open, usually due to jammed or sticking linkages and pedals. That's not a huge problem, since brakes are powerful enough to stop cars in this state. I don't think it makes a huge amount of sense to rewrite software to stop software-induced unintended acceleration, which probably doesn't even happen, instead of writing software for lane departure/collision warning/assisted emergency braking and other safety systems, which we know can help drivers avoid accidents.

I guess a third issue here is the American litigation system, which can turn companies into villains without the slightest indication that their products cause any more issues than anybody else's - all it takes is a non-zero probability of failure (true for most products) and a media + legal frenzy.

> The fact that no such recipe has been found, despite many millions of dollars spent on expert analysis, suggests that it doesn't exist in the wild.

I don't agree with this. Analysis of the source code, especially with global variables reducing the value of analyzing a unit, is not going to imply that it will discover anything about the (presumably millions of) users who exercise the code in live situations daily. Just the inability to CLEAR the source code from blame is a failing in responsibility on behalf of the brand.

The american litigation system has its issues, but I would argue that class action lawsuits are not among them. It allows social change through clear legal decisions when people are otherwise disagreeing on matters much like this.

The reason generally given is that in at least standard cars the brakes can apply far more torque than the engine can. It's telling how naive people are, they think the problem is 'unintended acceleration' A mechanical engineer thinks the problem describe is 'unintended brake failure'

And if the brakes won't stop the car, try the key switch! An engine that is not running will not cause acceleration. (Sure there are probably some oddball push-button ignitions that can't be turned off while in gear, but most vehicles can be.)

Turning off the ignition may deactivate your airbags* which could make a bad situation worse.You would be much better off attempting to shift your car into Neutral first.

* This was what made the GM Ingnition switch problem so deadly. The engine shut off (taking power steering and vaccuum-assist braking with it) and the airbags deactivated making any subsequent crash less survivable.

You're right that shifting is better. That is a pretty terrible failure mode for airbags, however. How much would it cost to put a big reliable capacitor in the airbag circuit, to keep the power on for 30 seconds? A dollar?

Absolutely right. After reading any number of stories about these "runaway death machines", what becomes clear is that the only realistic way for a car to accelerate at full power against the wishes of the driver is for the driver to be flooring the gas while thinking he is on the brakes.

We should be adding smarter automatic systems to the cars to assist drivers, rather than e.g. throwing out drive-by-wire linkages and replacing them with mechanics, which are generally less reliable.

There are already cars which will slam on the brakes in any collision which is strong enough to deploy airbags - it stops the car from careening around too much. It should be possible for cars to detect an impending collision and slam the brakes 1/2 second early to prevent or reduce it.

Reply on news.ycombinator.com