Hacker News Reader

I'm a member of the ACM (Association of Computing Machinery) and every year I read over the IEEE-ACM Software Engineering Code Of Ethics: https://www.acm.org/about/se-code

Due to manager and business pressure to make more money, multiple sections are subverted and there's absolutely no consequence to them. The only consequence they understand is if programmers start to refuse jobs or tasks or quit based on violations of the code.

This is sometimes why I would favour certification or more responsibility placed on the programmer. Some programs should not be written or if they are written they should be held to a very high standard. We wouldn't have issues with high frequency trading if it cost a lot to provide insurance for that kind of development. Sure simple websites don't really need high standards, but as soon as you're past 1 million in revenue you're dealing with real money, real data and any bugs could have serious consequences.

On the other hand, I like being able to write little scripts that say "hey if something breaks, it ain't my fault".

In the same vein is why I advocate for "Software Engineering" being brought into the realm of other engineers and having the status protected so people who call themselves engineers are held to the ethics and governing regulations that all engineers are.

Holding the P.Eng. designation myself I would be liable to lose my license had I published this software, I really hope we can reach a point in our profession where a higher standard of care to public and environnment is not only expected, but enforced.

I see this a lot. And I generally agree. However...

For software it's hard to see where to draw the line. For other engineering disciplines this can be pretty easy. I can release a little mechanical toy and not be an engineer. The public won't try to use it to drive around town.

But if I make a pet software project millions of people might decide to use it to "drive around town", and some portion will hurt themselves.

I think that if you build stuff for certain things (such as 1 ton objects zooming around at 70+ miles per hour), it's obvious. But what about financial software? What about social networks (privacy issues)?

When I first started programming I built a D&D utility for a friend. He passed it to his friends. Soon around 10 of them used it. If it had a serious bug, should I have been prosecuted? Should I have been unable to build it for him in the first place?

If you say all of this activity should be restricted, how do you get from here to there? Some of the best "software engineers" in the world aren't technically software engineers.

Software you write for fun or to pass around to friends would (should) have the standard disclaimer of implied warranty of suitability.

Software developed to control an automotive emissions system by a manufacturer and in the context of being used for that purpose would not be able to escape that.

This is a very common counter argument to software engineering as a discipline.

> But if I make a pet software project millions of people might decide to use it to "drive around town", and some portion will hurt themselves.

That is a legitimately unique aspect of software and deserves consideration.

> I think that if you build stuff for certain things (such as 1 ton objects zooming around at 70+ miles per hour), it's obvious. But what about financial software? What about social networks (privacy issues)?

Likely some portions of the system matter and some don't. Which ones likely require engineering judgement. For physical products it's pretty easy to make that call for a paper weight and for an airplane, but all the products in between need that decision. We are generally building software systems that are about as complex as a Black & Decker power drill. When designing something like that you need to make damn well sure you are not electrocuting the user, but if the gears break and it doesn't function you aren't hurting anyone. Therefore most drills are 'double insulated', but there is no backup for the actual function. In software we should make that same call. What is the consequence of failure? If it's small then we live with it. If it's not, then we should build a fail safe / backup system.

I'm super in favor of software engineering licensing and all that it entails, but I do think large swathes of software are completely immune because failure has no consequences. Your D&D utility is an example. Most open source licenses rightly have a clause that should protect them from civil lawsuits, much less prosecution.

> Likely some portions of the system matter and some don't. ....... but I do think large swathes of software are completely immune because failure has no consequences

This is where software seems unique - this seems false to me. A common failure mode of software is code execution, which has nearly unlimited consequences.

E.g. my D&D utility could read and write files. But it might have a bug in the file reading code (Edit: or anywhere in the code), which someone could exploit with a properly crafted file to gain access to the user's computer. But you said this utility was immune to failure. It seems clear to me that it isn't.

There is no physical object analogy here that makes sense to me. It would be like - if you used your black and decker drill on a piece of wood someone gave you, it could electrocute you if there were a flaw in the drill bit. Or it could turn off your refrigerator. Or it could play a song to you. This makes no sense, but this is the world software lives in.

Edit: or do you mean no physical consequences, so only software controlling physical devices require the proper licensing?

I was thinking more physical consequences. There's no way that a program running on my desktop can hurt me other than privacy / security / availability of my data. But I'm not going to the hospital because your D&D utility was written in a weekend and crashes sometimes.

But you do raise a point about the real costs of mistakes even in trivial systems. Arguably that makes the point that operating systems should have stronger sandboxes between apps. An operating system is effectively infrastructure, you probably want higher quality standards there.

Scott Alexander (of slatestarcodex) once wrote about medical ethics[1]:

> Listen up, this will be on the test. You need to help patients, not hurt them. You need to be responsible. Try to avoid being irresponsible. Try to act professionally, and to be a positive asset to society. Don't abuse nurses. If someone asks you do to something unethical, tell them you won't. Did I mention you need to help patients, and not hurt them?

> If reading that paragraph seemed like a waste of your time, imagine how I feel after attending the three hour lecture version.

Engineering ethics would be a similar waste of time. Good people don't need courses to know that cheating emissions tests is wrong. Bad people won't be swayed by a three hour lecture.

Here's an idea that would be more effective at reducing future harmful software: Run git/svn/hg blame to see who wrote the emissions-dodging code. Then arrest those people and charge them with crimes.

1. Against medical ethics http://squid314.livejournal.com/285785.html

That's sounds like an argument against "The problem here is people not knowing what's ethical." And accomplishes it, because the problem pretty clearly isn't that.

But the point still stands, that there should be some coordination among programmers to ensure that e.g. when someone is asked to do this, they have a group to fall back on, who's willing to help him/her out and boycott this employer.

If you don't have the spine to say "no, that's wrong, I won't do it" as a software engineer then I'm not sure a group is going to help you out.

Software engineers who are any good at all can find work easily. They are among the last people who should feel held hostage by a job. If your personal ethics are not enough, I don't see how a group will help.

Sure, if you're a teenager or 23 year old who shares an apartment and has no dependents and no debt.

Try that attitude on for size if you have a family that depends on your income, or you live in a city that isn't a tech hotspot, or you or your spouse or children have medical issues and depend on your medical insurance. You can't pay the rent or mortgage with ethics. There's lots of towns even in America where there are only a handful of high tech employers so no, you don't have alternatives. Unless you consider moving, which can easily cost $15k in moving costs and tens to hundreds of thousands of dollars in housing costs to be free. Not to mention the disruption to friends, family, and social networks caused by moving. Or care arrangements -- grandparents babysitting is close to free; daycare can easily be $400/week/child (or $600/week/child for under 2 y/o) in sf.

Have you really seen an employer not willing to pay relocation costs for software engineers? I haven't.

Yes (as would you if you bothered to look), and plenty more offer relocation bonuses of eg $5k which (1) will be taxed, and (2) are wildly insufficient unless you're basically living on a futon. For a family that needs a trip to house hunt and has furniture for 3-4 people, an actual relocation can very easily cost $15k ($3k airfare/hotels/meals/babysitting for 3-4 days in the new city to house or apartment hunt; $10k move for 3 bedroom house + car; apartment lease termination / house selling costs; other services shutdown plus startup in the new city).

I'm aware how much relocation costs. It's still less than what the developer costs you in a month.

I would love to teach a software ethics course, and it wouldn't be me standing up saying "be good" over and over. We would study contemporary ethics questions:

- what should you do if your government demands you secretly backdoor a specific user?

- what if your easiest recruiting channels yield a higher proportion of white males than other, costlier recruiting channels do?

- if your startup is struggling to even survive, should you invest time in infrastructure that protects users' privacy from your employees?

- how do you find nonprofits and companies that are aligned with your values and need your help?

Those are just four questions off the top of my head. I think there are plenty of thugs to talk about to fill a semester, let alone an hour.

I would pass on attending your class, but I wouldn't mind watching a feed of the proceedings. The fistfights would be especially entertaining.

> - what should you do if your government demands you secretly backdoor a specific user?

They've got a warrant or a court order? OK.

> - what if your easiest recruiting channels yield a higher proportion of white males than other, costlier recruiting channels do?

If you're a good person, you try to correct for that. If you're a money-grubbing sociopath, you try to correct for that. This may sound unusual, but think about it from the sociopath's point of view: If other companies are neglecting some potential candidates, you can hire those people for less pay. Recruiting is cheap compared to salaries, so it's worth the upfront cost. In the long run, even if everyone is a money-grubbing sociopath, all demographics end up being paid the same.

> - if your startup is struggling to even survive, should you invest time in infrastructure that protects users' privacy from your employees?

Probably not. A dead company can't defend users' privacy very well. Also, assets (such as user data) tend to get sold off when a company shuts down. Best is the enemy of good, and all that.

> - how do you find nonprofits and companies that are aligned with your values and need your help?

Givewell.[1]

These are not hard questions to answer. More importantly: Your hypothetical course, even if taken by every software engineer, would not have prevented the Volkswagen emissions cheating. That was obviously wrong, but the people involved didn't care.

1. http://www.givewell.org/

I believe this falls under the "Louis Armstrong Rule".

If you have to take a class on ethics, you don't have them.

(Q: What's the "Louis Armstrong Rule"? A: Man, if you have to ask, you'll never know.)

If you have to take a class on math skills, you don't have them.

I fail to see what your point is.

If you already have enough ethics, you can work out on your own what would be ethical for any given situation. You don't need the class. It may save you a bit of time in reflective contemplation, but it's not going to give you anything you didn't already have.

If you don't have enough ethics to figure it out yourself, the class is just teaching you how to pretend to have ethics. And like jazz, you will fail just as soon as you are called upon to improvise.

To me, there is an ethical "operating system" which is what you are describing, very low level principles that you apply.

But your ability to be ethical in a specific situation depends on your database of "ethical considerations" about possible decisions you might make that will have serious ethical ramifications and how those things are wired up. The purpose of a course is to build up that database.

You might have a deep and earnest desire to be kind to someone, but without a cultural framework for understanding their feelings, for example, you could be completely unable to do that.

The point isn't ethics courses.

The point is institutional protection for engineers who are ordered to do unethical things and refuse.

I tend to think that our society should move in the other direction. I see the need for college degrees and certifications as a form of caste system which limits upward mobility and increases cost of living. I would like to see education and certification added to the list of qualities on which you're not allowed to discriminate (like race/creed/etc already are).

I think the VW case can be partially solved by handing out indictments to everyone involved, CEO down to engineer. Working in a corporation does not excuse criminal behavior.

Similarly, I don't think making Financial Engineer a thing would do anything to head off the next financial collapse and reign in excessive risk taking on the part of the banks.

Education only resembles a caste system because its availability is heritable. The quality of public education depends on the income characteristics of the geographical location you're born into; the quality of higher education depends on the disposable income of your parents (except at the most elite schools with full need-based scholarships for middle-class students).

I'd rather see us abandon the bizarre notion that education is in any way a parental or local responsibility. It should be federally controlled with uniform quality for every student and differentiations that make sense (on natural talent, effort, and specializations chosen) rather than those based on parentage.

Good education gives you a better understanding of what you're doing and the context of the world, people, and systems you're doing it to. Of course it makes a difference, and it should be available to everyone.

It's not discrimination if you can change it.

The difference between someone with a college degree and without is the work they put in. That seems like a pretty reasonable thing to base a hiring decision on. Your suggestion is like saying 'you shouldn't be able to discriminate based on work experience'. The difference is college is 4 years of reading books and taking tests vs 4 years of writing code.

You can argue which one is more valuable / relevant / useful, but they're just different kinds of experience. Not allowing that in a hiring decision is flat out unworkable.

You can also change your religion, marital status, and parental status. Those are all protected from discrimination.

College isn't just a matter of putting in work. It's a matter of putting in money. That is the clear dividing line versus work experience.

We have it in Portugal.

http://www.ordemengenheiros.pt/en/about-us/colleges-and-fiel...

However, since the law doesn't require informatics to be inscribed for daily work, most don't.

> Sure, it may have been a PHB who came up with the idea and told them to code this into the algorithm - but it was programmers like you or I who actually cranked out the code that made this possible.

So i think there's been a lot of assumption that the software engineers made this code explicitly for production. For all we know they were making what they thought was code for specific in-house experiments and a subset or different set knowingly kept it for production.

Actually that was the purpose stated; they developed it for testing in-house and it makes sense to do so. However, at some point there must have been someone who was a front-line employee who could have blown the whistle on this kind of disgusting ethics violation.

That's possible, but I find it to be a stretch. Unless one group were told "write some code to meet this emission standard" and another were told to "write some code to detect when emmission testing is happening" that could be the case. But somewhere along the line someone was given the task to combine those and "disable the emission controls unless testing is happening."

In other scenarios, yeah it's perfectly possible for an developer to complete a task that seems innocent enough and unbeknownst to him will later become a part of something nefarious.

The engine control software in question was produced by Bosch. They say they warned Volkswagen that the feature in question was for development only and not to be used in production.

It seems unlikely that Bosch would implemented an algorithm to detect an emissions test and alter the engine behavior. It seems much more likely that Bosch implemented diagnostic features to force the engine into a particular operating mode (e.g. Enable/Disable EGR) and that Volkswagen activated those features when it detected certain qualifying criteria (i.e. emissions test conditions).

Unless there's direct evidence implicating Bosch, then blaming them for the cheating is like blaming a bullet manufacturer when you get caught shooting people.

Was it a stern warning, loaded with gravitas, or was it accompanied with a wink that passed unnoticed by the corporate attorneys?

Bosch might have sold them the pistol, but VW is still the one that shot itself in the foot with it. But was the pistol pre-loaded with toe-seeking bullets, or did VW carefully aim at its own extremity before firing?

There is still some question in my mind as to how complete a defeat device package Bosch shipped. If there were several separate functions like LowestEmissions() and LowestFuelConsumption() and LowestMaintenance() and BestPerformance() and LowAmbientTemp() and HighAmbientTemp(), that's quite a bit different from DetectEmissionsTest() or OperateAsDefeatDevice(). Who was it that wrote the code to switch modes when an emissions test was detected?

It's not accurate to compare this to a building falling down.

Nothing went wrong, the code did exactly what it was designed to do.

From what I understand of the situation, It's entirely likely that the devs got a rule table from the engine and emissions guys and couldn't actually tell if the metrics were good or sketchy as hell.

It sounds like the main problem is with high fidelity O2 sensors at a cheap enough price point that these lookup tables can be banished. We legislated people carrying a block of platinum around in their car. Maybe $200 O2 sensors is the next step if the manufacturers can't be trusted.

There's a whole business structure that drove and enabled this decision, it wasn't just a bunch of engineers who had a bright idea one day. (Not meant to be exculpatory for the people who worked on this.) But you never hear anyone going around calling for businesspeople to conduct themselves ethically.

Well said! I did think about the programmer or programmers when this story first broke. I am not shocked that some executive would come up with the idea...but I am shocked that someone would actually code it. You have to wonder how many people knew and said nothing.

I could imagine a scenario where this started out as something less evil. Seems to me that cars will perform differently in a lab than on the road; off the top of my head: - Stationary cars have different air-intake characteristics - The temperature of the fuel and air is likely different - The humidity of the air might be different - The fuel in the tank is stationary not sloshing around

There may well be valid reasons for the software to vary engine parameters test mode; maybe to compensate for the above.

Then sometime later someone said "This NOx Reduction module is killing fuel efficiency on the open road, how about we turn it off"

The net result is the same but motives and intentions of the individuals are important.

What steps do people take to comply with #2d ? Does the British Computing Society maintain a service which keeps people up to date with legislation and relevant court rulings and where you can ask hypothetical questions of actual lawyers?

This one does seem quite onerous: how do you even know what legislation applies to the software you're working on, to even ask the right questions? There are already specialists in legislation (we call them lawyers), so why should the programmers take on this responsibility?

I agree with the spirit of the rule, but I don't think it makes sense as it's basically asking all programmers to become lawyers. It should be more along the lines of "comply with legislation you are (made) aware of" and require some due diligence like "solicit the services of a lawyer if there is a reasonable belief there is legislation that could apply".

That's part of your professional responsibility. How do you know not to store passwords in plain-text? How do you know that you have to make your software accessible to people with visual impairments?

I honestly don't think you need a legal degree to know that "dramatically alter how the pollution controls works when in a test environment" is likely to be extremely dodgy!

Now, maybe these programmers did flag this as a problem. And, maybe VW's high-priced lawyers said it was OK. In which case, responsibility is probably shifted.

I'm not saying every programmer should be a lawyer. But you can't say "how was I supposed to know that running a red light while drunk is illegal?! I'm just a delivery driver - not a lawyer."

I do receive emails from them talking about policy changes - and the BCS's position on government consultations.

They also have a number of reports & white papers which members can read.

Members also have access to a legal helpline. Don't think I can link to it, but it says...

> As part of the ongoing commitment from BCS to invest in services and benefits, the BCS has launched a new and improved Legal Advisory Service comprising two new online legal information websites and a dedicated telephone helpline for both Business and Personal legal issues (restrictions apply*). These services are totally FREE and are provided by Law Express, the UK's leading specialist provider of legal advice.

I've not had cause to use it, so I can't comment on how well it works in practice.

Do we have the bad VW code? Do we know it happened how you allege?

Do we know that the code was written by hand and not auto-generated based on real-world testing/calibration?

Do we know that a programmer has final say on the code that is shipped?

Are you sure management couldn't select from a number of programs based on the criteria they like, without needing additional input from a programmer?

Why didn't management review the codes and put in place procedures to ensure the codes reviewed are the ones actually being used?

Why didn't management implement testing procedures so that the reviewed codes could be verified?

Do you have the specs/requirements the programmers were given?

Edit: Really? Downmods?

Your story may not be 100% accurate, but still, it may be a good idea to introduce a "Hippocratic oath" for programmers, or engineers in general.