Hacker News Reader

I have to disagree. Resorting to "keep it legal" removes any moral obligation. There are ethical decisions to be made, some easy, some not so.

Should IBM not have worked on Computers? No.

Should IBM not have sold to Nazi Germany? Most likely yes.

Should the guy working hard all day to make the Zyklon B have refused to do so? ABSOLUTELY

Back when I went to school, we had one (or two?) hours per week of "ethics class". If there was one thing to take away from those lessons, it's that you can never just refuse to exercise moral/ethical judgement. Acting ethically is one of the central responsibilities that a citizen in a free society has.

> Should the guy working hard all day to make the Zyklon B have refused to do so? ABSOLUTELY

Initially at least, Zyklon B was an entirely legitimate pesticide/vermicide that had a broad range of uses. The original form contained a powerful warning odorant to alert users of its presence in sub-lethal concentrations.

I don't think there's any claim that the actual production staff knew exactly where or how it was being used, and so would have no real grounds to refuse or quit. There is evidence that once it was found suitable for killing humans, there was a change in formulation to remove the warning odorant at the direct request of the extermination camp officials.

I'm not sure if that alone would be grounds for staff suspicion, as it was apparently passed off as necessity due to wartime material shortages, which to me would seem plausible.

The production and distributor firms management, however, knew a lot more about what was going on, and where it was being used. Founder Tesch & director Weinbacher were both found guilty in a war-crimes tribunal[1], whilst another employee was acquitted.

So I don't think it's quite so simple a scenario as 'the process engineer for the pesticide vats should have guessed they might be being misused and quit just in case' that you lay out.

[1] http://www.ess.uwe.ac.uk/WCC/zyklonb.htm

...Which is exactly why engineers (like me) simply must seek out and look beyond the Kool-Aid their company and managers pour and understand exactly what effects the actions of their products and services have on the world.

We shouldn't just come into work, pull a bunch of levers and tell ourselves, "I'm sure the managers wouldn't let this happen if it were evil." We have to leverage our curiosity to extend beyond the Disney version.

BTW, this goes equally for marketers, a group I feel is as bad at ethics as engineers. Persuading people to continuously purchase and consume unhealthy food and activities is as destructive to the human spirit and our nation as bomb-making.

</soapbox>

Zyklon B was invented in the 20s, long before WWII. So I'm not sure what Kool-Aid they were supposedly drinking. There were many legitimate uses.

Additionally, Haber (of Haber-Bosch, indirectly responsible for feeding like what, a third or more of current human population?) deliberately worked on chemical warfare as he really believed "in Germany". I'm pretty sure he didn't see his work as evil, and likely many of his employees would agree. I think I recall that they hoped it'd be such a shock, it'd end the war earlier.

I agree with your sentiment in general, but things aren't as easy as you're phrasing them.

For anyone interested in more about Zyklon, Haber, mustard gas, nitrogen based fertilizer and the ethics of it all I highly suggest checking on the radiolab episode that tells their story. Here is the relevant section: http://www.radiolab.org/story/180132-how-do-you-solve-proble...

But indeed the workers producing Zyklon B in large quantities had very strong indicators available regarding what was going on.

Think about it: You produce this poison gas for years and then, suddenly, a large part of the production pipeline is used to produce that same gas but without the smelling safety compound. What possible plausible reason could there be except using it against people?

I admit that some of them might have suspected that they were working (illegally) on a secret stash of poison gas for use in warfare. But that makes it still morally wrong.

If you have already listened to the Radiolab episode on Haber, I recommend also the one on the Milgram experiment. They, unlike practically every other instance of Milgram explanation out there, reported the actual results that the experiments gave: People will _not_ do evil when ordered, but they will do evil if they believe it is neccessary and serves a greater good.

Back to the workers.

Most of those bastards probably suspected something but found it unsetteling to pursue this mental avenue, so they didn't.

Some asked questions or raised the issue with a superior and recieved an evasive answer. Sensing uneasyness ahead, they did not pursue the issue.

Finally, if there were those that raised moral objections, they would have had a talking to, not a harsh one but an understanding one. "We know this is hard for you." "It is part of the bigger plan for Germany" "Someone has to do it" "We are all indebted to you for taking part in this gruesome but important endeavour".

Humans are like that. These guys all chose the easy way out. But it was still wrong.

"Every algorithm known to man has both good and bad uses. Not developing algorithms because they might be harmful is insane."

Yep, that missile guiding system someone built for Boeing totally has primarily good uses in mind.

"Work is just work, a means to exchange your talents for money. Keep it legal, do good work. Go home and play with the kids."

Dude, seriously? Everyone should have some sense of morals and question authority. Come on, if we don't question our current way of life, how does anything get better? Why should we be okay with people dying just so we can have a nice, cozy life with our kids?

"Dude, seriously? Everyone should have some sense of morals and question authority"

Yes seriously. My "sense of morals" is very different from yours (in that I'd gladly work on defence tech say.). So what?

I would gladly work on "defense tech" too, as in actually defending people. But what you mean by "defense tech" is actually offense tech. Look at how these things are actually used in the world.

When an entire field needs to stand behind a euphemism in order to prevent seeming ugly, that is certainly some kind of a sign.

There is no difference between a technology being used defensively or offensively. The 'self driving car' tech will feed into 'self driving tanks' tech, and vice versa. Military Satellite tech gives you the GPS and Google Maps, which will in turn be used for military purposes. A technology (including algorithms) can be used for good or ill. That is just the nature of the beast.

You can certainly work towards what you see as the overreach of American militarism. It is a political/economic problem which needs a political/economic solution. Your (not) working on cryptanalysis algorithms has nothing to do with it.

To play devil's advocate: Let's say my morals aren't bothered by killing members of the Taliban / Al-Qaeda / Enemy Of The Week, as I consider them a bunch of fanatics and a threat to western civilization. As such, I work on guidance systems and weapons to try and improve their effectiveness and accuracy, making them more precise, and less likely to kill civilians and cause collateral damage when employed.

Now what? Where do you derive your morals, and what makes them inherently correct, and mine reprehensible?

The missile guidance systems used for targeting offensively is similar (not the same) to the missile guidance systems used for intercepting incoming ballistics.

Are you then suggesting that guidance systems for missiles that intercept (like a thaad or mim patriot) should not have been created?

That is one way things are actually used in the world. I would think a case such as that could be seen to be morally ambiguous.

The primary purpose of that missile guiding system is to miss civilians.

Unguided bombs are very cheap and highly effective at taking out the target.

http://en.wikipedia.org/wiki/Bombing_of_Dresden_in_World_War...

These days, the primary purpose is quite likely to be taking out hardened targets of one kind or another - unguided bombs are not terribly suitable for that, unless you use a lot of overkill (as in, nukes against low-tech adversaries hiding in caves overkill, most likely).

Where do you think GPS came from?

I'm saying people shouldn't actively engineer things that will hurt people. If every engineer refused to build weapons, there would be no weapons. GPS would exist without the army. If someone is putting GPS into a homing missile system, then they're actively building something to harm someone else.

No, it wouldn't exist, actually. Do you really think the government spent $100s of billions so you could navigate in your car?

It possibly wouldn't have existed until a much more modern era when the cost would have been less, sure. But someone (Google hint hint) would definitely invest in GPS if they thought it would make them money. Besides, I don't think you can really compare GPS with say, making explosives that can reach a greater blast radius. My point was that to simply say "well, some Army projects can be used for bad and for good" does not justify the engineering of technologies that are clearly meant for killing.

when you say "keep it legal" it sounds like you only mean, "so I don't get arrested". That is, you would not work on something very ethical, but illegal (like some system of assisting in the production of medical marijuana in a state where it's illegal) but you would work on something unethical but legal (NSA spying).

So really, saying anything legal is not saying much - essentially ethics don't come into question.

I know for a fact that I'm very uncomfortable working on something that I find to be obviously unethical. Though NSA stuff, I'd not work on that just because I'd hate keeping everything I do a secret.

Sure, as with all things human, there is the occasional gray area where there are no good answers, but yes I probably wouldn't work in a criminal enterprise in a democratic nation, whether it is about medical marijuana or not.

The point is, I'll work on things as per my ethics, not some random person's ethics or a hypothetical "engineers code of conduct" or whatever as the article seems to advocate.

I gave plenty of examples of software that some people would consider unethical (say drone targeting, or NSA crawling etc that TFA is about) that I would have no problem working on. My ethics are mostly in harmony with the legal systems of democratic nations. I'm sorry if that doesn't accord with your notions of what "should be".

Oh well that's something different - you'll work on what you find to be personally ethical. That's absolutely fine, ethics after all are ultimately personal. Like I'd not work for Monsanto, since I find them unethical, but certainly most hackers disagree with me on that. I'm not happy about the NSA situation though I see it more as an issue of bounds being overstepped, the engineers working there don't strike me as empirically unethical.

I agree completely. I threw in 'legal' in the original post to head off morons chanting "so you'd work for drug dealers" etc. My only point was, to quote you, "the NSA situation ...I see it more as an issue of bounds being overstepped, the engineers working there don't strike me as empirically unethical" (well stated).

I have a hard time listening to the demonizing of the NSA engineers by people who work at Google/Yahoo/Facebook/Amazon (or would like to) who build similar technology to profile users!

Throwing in "legal" in a discussion that pretty much revolves around that seems like a poor way to communicate.

Also "drug dealers" is probably the epitome of a morally/legal ambiguous job. Purdue has no doubt eased the suffering of millions of people, yet they were still hit with felony convictions.

> The point is, I'll work on things as per my ethics

I'm a bit confused... are you saying there are some jobs you wouldn't take no matter how good the salary was? You seemed to imply otherwise in your original comment.

oh what would we do without absolute black and white with sharp dividing lines ? Go read grellas's comment. He says it much better than I could

I did read Grellas's comment, and I'm not sure why you think my question had anything to do with "absolute black and white with sharp dividing lines."

In your original comment, you said:

> give me enough money and challenging work, great working conditions, and I'll work for any legal company or organization, though some organizations would have to put out a great deal of money vs others

The key word here is "any." You later stated that you operate by your own ethical standards, and didn't seem to disagree when zzzeek interpreted that as "you'll work on what you find to be personally ethical," so I was merely asking you to clarify if you would really work for "any" legal organization.

Of course, if your ethical standard = whatever the authorities deem "legal," then your position is entirely consistent.

There is a point when further nitpicking communication is not only useless, it is actually a disservice to the forum. I leave the thread to you, and bow out. Cheers, have a nice day.

I honestly didn't think I was nitpicking, just trying to gain some clarity on your moral perspective.

Also, in terms of grellas saying it better than you could, he specifically notes that we "ought to avoid being a proximate cause of something deemed wrong even though technically legal." This directly contradicts your comment, in which you implied that you would work for any legal organization if the money was good enough. I was merely asking if you could address this contradiction, in the spirit of provoking reasonable discussion about the intersection of money and morality. But if you feel my line of inquiry was nitpicking, then I apologize.

Hey no need to apologize, it is all good. What I meant by "go read grellas' comment" was "Ignore what I wrote, Grellas says what I should have said".

Yes the use of the 'legal' bit (which I threw in to keep the ravening hordes off 'but but are you saying you are ok with drug dealing/child pornography/whatever'. In my defense, it was late at night here (India) when I wrote that and it is too late to edit now.

My position in my original comment was "Within the law (see below for an alternative to this clumsy phrasing), I'd probably do almost any job in software, provided I were paid enough". The "enough" might be quite high.

As a thought experiment, take a sw job you'd find distasteful, and ask yourself if you'd take it if your pay was a million dollars a year. How about if it were a billion? If you are the rare human being who wouldn't work for the NSA, no matter what you were paid, then more power to you. I would gladly work for them if they rewarded me well, because I don't think the engineers at NSA are evil demons bent on world domination. By that logic, nobody should be a soldier, because the essence of that job is killing other people for dubious political causes. Yet the USA worships its service personnel ("Thank you for your service") to a far greater degree than most other countries (where being a soldier is "just" another job).

Grellas of course lays all of tis this out much better than I can. "proximate cause" is the phrase I should have used, but I am not a lawyer and wasn't even aware of the phrase.

So, yes I probably wouldn't work on something that has a pure unadulterated horrifying evil proximate effect( and no good aspects) to it. But that isn't saying very much since this kind of pure evil job probably doesn't exist outside a platonic ideal.

I was objecting to the broadbrush sanctimony in the article, and the use of (in my mind) silly examples.

The whole "legal" bit was clumsy phrasing on my part and you are right to call me out on it.

Cheers,

Well, I guess I wouldn't work for the NSA no matter what (well, it depends on the specific job, of course, they might actually have some positions that are not ethically questionable, after all), and there is one important reason why pay cannot compensate for it: The effects are not just on others, but also affect myself, so that's a cost that I have to subtract from the pay. A society with complete surveillance removes everything from life that makes it worth living, for everyone, including myself, and you can not buy that back, so money would be worthless as compensation.

I think a common misconception is that ethics are somehow something that you obey for others to gain from it. A concensus of behaving ethically creates wealth for everyone, including yourself. It's not that we do not commonly mug other people in the street because we don't want to have their money, but because a society where you can expect to not be mugged is just so much nicer to live in, so that is a value in itself. The only problem with that is that society can support some free riders, and so there is some motivation to be the free rider.

I can't agree more. Legal isn't the same as moral, and conflating the two is pretty disingenuous.

My professional goal (hell, my life goal) is to work on things that make a positive difference in the world. Laws don't necessarily come into it (and sometimes are in direct opposition to this). I believe (or naively hope) that other people in my profession feel the same way, otherwise the future is going to get... interesting.

Interesting attitude. Why "keep it legal" if this is your opinion? (besides the risk of getting caught).

> Keep it legal, do good work. Go home and play with the kids.

Tell that to the people whose kids got blown up at a wedding by a Hellfire missile that was guided by your TwitterTerroristLocator.so

"Legal" is exactly what unethical behavior has been hiding behind. Setting your ethics to "legal" is weak.

I used to think working for the NSA would be fun, interesting, and in the service of my country. I don't think that anymore...

"great working conditions" for me implies that I don't find the work odious. No amount of free soda can make up for having to work on a shitty or immoral product.

This is a luxury afforded me by the decent amount of decently well paid work available to an engineer. I might have less moral freedom if I couldn't feed my family.

Having any ethics whatsoever is "political correctness" now? I don't think you know what that word means.

It sounds like you simply allow the law to serves as your morals.

This is called a lack of artistic integrity.

And there's nothing wrong with that, you're just excerpting yourself from responsibility, and if enough people do that engineering becomes irresponsible.

Arguably this is a problem right now.

Look, I understand your position, food has to get on the table, but: I don't respect you. I am responsible for everything I put out in the world.

You can take that as a guilt trip if you'd like, but I feel like if you're going to be bald-faced about how you put all the real responsibility on laws and those who make them, I can be blunt about how I view your attitude.

I guess everyone has to find their own relationship to the value of their work.

" but: I don't respect you."

heh sure. whatever makes you happy.

Heh. Nothing left to say now, really. Cheers.