Comment by coldnebo | Hacker News Reader

Comment by coldnebo | Hacker News Reader

Yes, but in this case compliance is quite difficult. Why would the compliance department go out of its way to develop a new mobile testing infrastructure when that isn't the industry standard? So it seems very plausible in this case that compliance might not have found the issue. It required a novel method of testing to discover the cheat.

I have to be honest, when I first heard about this, I suspected a genetic algorithm as the culprit. It's exactly the kind of result you'd expect when giving an impossible optimization without sufficient boundary constraint. AIs can and have been known to cheat, and because the fitness driver would have been compliance (V&V) the GA could plausibly have settled on methods that were undetectable (i.e. Not part of the fitness function).

Often the output of GA code is not understood formally by engineers, or reverse-engineered into a traditional algorithm, so it's plausible that it could have remained hidden in plain sight.

BUT, if that had truly been the case, the engineers at VW would have quickly suspected and publicized it. So the fact that VW blinked showed that in fact they lied and didn't have an honest technical explanation.

However, I think the GA scenario is good to keep aware of as we increasingly rely on machines to optimize solutions based on testing.

The "sin" they are accused of it explicitly detecting that certain test equipment is connected. That's a binary, if-then-else sort of thing. "The boss is looking; look clean."

A GA (or any machine learning ) within a controller is less about being able to cheat than it is ...rebasing something that would otherwise be a constant (from the frame of reference of the algorithm) in a control-feedback arrangement.

If the GA was part of a test stand and exhibited emergent... dishonest behavior :) then ... they did it wrong.

There are valid reasons to write code to determine if your vehicle is being tested. Modern cars contain all sorts of computerized safety features that would go active in the typical test scenario thereby preventing the static tests that the EPA and others do.

So the car detects wheels straight, only drive wheels turning, and so on, so it disengages the features that would active traction control, reducing engine power, and more. The key here is that VW engineers went further and used that as they indicator to run the full emissions process and only then. There is no viable reason to have any toggle on emissions processes except perhaps with regards to cold weather starts.

We think it's "run the full emissions process" but we're guessing. It's a very good guess, but it's essentially unsupported. It could have been an interaction with one of the other things that got turned of. This seems unlikely,. but it's not 0% probability.

Doing GAs right can be very hard. Like, I've seen cases where GAs hooked up to a certain type of physical simulation learn to exploit the ODE in play and find optima that are based on ODE artifacts, not reality. Eg wind turbine design and CFD modeling.

I remember one specifically learned in Quake to always strafe run because the dx+dy gave sqrt(2) max speed instead of just max speed.

What most people don't understand about GAs is that asking the question correctly and unambiguously can be harder than solving the problem. GAs can have a steep price, it's not magic.

> Yes, but in this case compliance is quite difficult.

Agreed. It's highly unlikely that the Compliance department could have detected what the engineers had done, short of having someone audit the code, line by line, or inventing a test to check the emissions while the car was moving. Based on my experience in the financial sector, Compliance departments don't work like that - if someone really wants to cheat or circumvent the rules, there's a good chance they'll be able to conceal it from Compliance, at least for a while.

People are responsible for their own actions, even when their job title is "engineer".

VW modeled specifications and tested them for other components.

Why wouldn't these components be written to specification?

We're not talking 1.1 vs 1.2, we're talking a 40 fold increase. If the compliance department is not able to detect such a massive fraud then they should all be out of jobs. And if they don't audit the code then they should be out of jobs anyway. There is a big difference between process compliance (the kind that the financial world is burdened with) and product compliance. Independent laboratories would have been able - and in fact did - to find these discrepancies with comparable ease, further down there is a link to a magazine article where 9 cars were tested by an outside lab just to write an article.

The magnitude of the increase is irrelevant. The key question is whether it was reasonable to expect them to detect it, which is, in turn, informed by the question of how they could have done so. The obvious method to audit compliance against an emissions standard is to use the same test that the regulator does. Obviously, in this case, that wouldn't have detected the defeat device.

To detect it, they would have needed to either conduct a non-standard test, or audit the code. Both those solutions seem, to me, to be above and beyond what would be considered reasonable due diligence under the circumstances. Perhaps I'm wrong, though - perhaps other car-makers perform such tests as a matter of course? I'm happy to be presented with evidence that that's the case. Even if it is the case, however, that still doesn't absolve the engineers from blame.

The bottom line is this: You are suggesting that the engineers who created the defeat device should not be blamed for doing so because the Compliance department should have detected it. I'm saying that not only was it not necessarily reasonable to expect the Compliance department to detect it but, even that were the case, it doesn't absolve the engineers of responsibility for what they did.

I suspect the engineers were ordered to do this, not that they came up with this by themselves. (In fact, I find that suggestion totally unbelievable.)

I agree the engineers are culpable but if they were acting under orders from their managers then the buck has to stop somewhere, someone is going to be the last one 'up the chain' that decided it was within their pay-grade to make this decision. The internal organization of VW is such that that decision could have only come straight from the top given the liability in case of exposure.

If the only way to ensure compliance is to audit the code then that's what you should do. It basically says the testing regime is so easy to detect that it can be used to defeat the test itself, Bosch correspondence indicates that this software is expressly not meant to be included in the final product and VW knowingly ignored that information, something a compliance department should have been absolutely on top of.

At some point, somebody came up with the idea and a decision was made, either individually or through consensus, to use the defeat device. Why do you find it so unbelievable that it was an engineer (or a group that included engineers)? Would it make a difference if it was an engineer who had been promoted to be a team leader? Promoted to "chief engineer"? "Engineering manager"? At what point would the person's title be far enough removed from "engineer" that you would find it believable? Does having the word "manager" in one's job description automatically make one less trustworthy than someone who has the word "engineer" in theirs? Shall we just adopt a policy of "Engineer good. Manager evil."?

You say that the compliance department should have audited the code. Do other car companies' compliance departments audit their code? I doubt they do. I suspect it would be cost-prohibitive.

> At some point, somebody came up with the idea and a decision was made, either individually or through consensus, to use the defeat device.

What should have happened:

Hey boss, there is no way that this engine is going to pass emissions. What should we do now?

Manager: sorry, but if there is no way, there is no way, I'll communicate that up the chain.

What possibly happened:

Hey boss, there is no way that this engine is going to pass emissions. What should we do now?

Manager: let me confer with upstairs.

Some time passes.

Manager: there is no such thing as no way let's find a way, cheat if you have to (but don't tell me you did).

Or something even more damning.

Now, the big question is what happened during 'some time passes'.

Whether the buy-in went all the way to the top or whether that person made this up all by themselves. VW wants us to believe the second. Knowing a bit about German corporate culture I sincerely doubt that. If there is one thing that Germans excel in it must be strict adherence to process and authority.

> Why do you find it so unbelievable that it was an engineer (or a group that included engineers)? Would it make a difference if it was an engineer who had been promoted to be a team leader? Promoted to "chief engineer"? "Engineering manager"? At what point would the person's title be far enough removed from "engineer" that you would find it believable? Does having the word "manager" in one's job description automatically make one less trustworthy than someone who has the word "engineer" in theirs? Shall we just adopt a policy of "Engineer good. Manager evil."?

Engineer is a title that comes with a whole bunch of responsibility. Manager and Engineer are not mutually exclusive titles. But clearly there is a line drawn on the VW org chart that supposedly insulates those higher up from any fall-out from this other than some 'formal responsibility' while using the old and tired 'but I didn't know about it' line. I don't buy that for a second, you just can't have a thing like this going on in a car company that has the ambition to be the worlds largest supplier of vehicles and not be in control of the product that is being produced. That spells 'corporate cultural rot' to me at a level that would be far more damning than outright malice. Of course outright malice might end with direct culpability for management which is why we get this other spiel.

> You say that the compliance department should have audited the code.

Yes.

> Do other car companies' compliance departments audit their code?

I would bloody well hope so. If they don't they ought to get other jobs. Reviews are there for a purpose, it is not that this stuff is so different from one version to another that you could not spot the commit that is tagged 'added test cycle detection and defeat code' (or even if it is mis-labeled on purpose to spot that someone inserted a bunch of code with a description that does not match).

I worked for a bank for a bit and there was absolutely no way that I'd be able to get away with a trick like this without being caught.

> I doubt they do.

That could be right.

> I suspect it would be cost-prohibitive.

This would hold true for Morgan (who buys their engines from BMW and they're not diesels) but it certainly would not be true for VW, the cost of a thorough audit would be an extremely small fraction of the cost of the fines they are now incurring and on the budget of VW it would be next to nothing.

What possibly happened (alternative version):

"Shit, I don't think we can hit these performance targets and stay under the emissions limits."

"Hmmmm... I think you're right."

"The boss is not going to be happy. I guess no bonus for us this year!"

"Hmmm... What if we wrote some code for the ECU to detect when the vehicle's being tested and dial down the performance?

"What, are you serious?"

"Yeah, that way it'll still meet the emissions standards, and hit the performance targets, right?"

"No way, man. Das ist verboten, dude. The boss'll never go for it!"

"He doesn't have to know! Nobody has to know. We just slip the code into the ECU when nobody's looking. Nobody's going to check it!"

"I dunno, man..."

"Come on! You want that bonus, don't you? Think of all the lederhosen you can buy!"

Paul Kedrosky wrote about this type of scenario: http://www.newyorker.com/business/currency/an-engineering-th...

I've worked with quite a few German companies, none as large as VW but this is absolute SF (entertaining though it is). The Germans have a word for their way of working 'gruendlich' ('thorough') and they (usually!) mean it. How this happened will be the stuff of endless case studies when - and if - the truth will finally be on the table. At a guess that will not happen before all the current board members and the former CEO are well in their graves.

BTW the whole scenario outlined in your link is bunk. The code came integral from Bosch and was definitely not developed gradually. It basically is designed purposefully to do an end-run around tests and Bosch explicitly warned against inclusion in the production version of the software saying 'this would be a very bad idea'.

If a single engineer developed the engine, including the software, it would be quite likely that he or she came up and implemented the cheating solution alone.

But that's not how things work.

Imagine being in the team that develops an engine. Some prototypes are built, measurements are done, way too high emissions are noted. Who gets the results from the measurements? The project lead, of course. Then the lead calls in folks from different parts of the team (or even different teams involved), and they brainstorm what to do. This becomes a big topic, because it puts the whole project at a risk.

If at some point the problematic values simply disappear, any diligent project lead has to investigate where the improvement comes from, if only to make sure there's no regression.

How would the scenario look like so that the project lead never even noticed? The environmental testing reports directly to a software developer (but why would they?), who implements the cheating part. Without first asking the rest of the team if there are legit ways to accomplish lower emissions. All documents regarding the first test runs vanish (but why would they? it's not the tester's job to make sure the emission standards are being met), and the problematic emissions never become a topic.

Somehow that sounds not realistic at all.

So, I'm pretty sure the project lead knew about it. the project lead might have been an engineer by training, but his role was that of a manager.

The engineer who actually wrote the code was most likely ordered to do so, yes. ("write something to set a flag when the wheels are running but the hood is open, the GPS is silent and the accelerometers don't detect anything stronger than someone tapping at a window")

Most likely, that engineer was ordered by drumroll another engineer. If you trace up the the chain of command at VW, it's not unlikely that it is engineers all the way up until you reach the shareholders, and even there you will find a significant fraction of shares owned by engineers.

VW has drawn some fairly arbitrary lines on their org chart where the 'problem' is located, namely anything down from and including the head of engineering.

> The obvious method to audit compliance against an emissions standard is to use the same test that the regulator does.

Sure, but those folks also have common sense, and experience. If engines with comparable power have 40x more emissions, you'd do well to understand where such a huge change comes from.

I've done some measurements as well as simulations in physics, and one of the first things you learn is to be suspicious of outliers, both of results that look like total garbage but shouldn't, and of "too good to be true" results.

The only way not to raise a red flag would be if all of the compliance engineers were new at their job. And that itself would be suspicious too.

Reply on news.ycombinator.com