Comment by DoreenMichele | Hacker News Reader

Comment by DoreenMichele | Hacker News Reader

Parent Full thread

I have really mixed feelings about this. To my mind, the larger outrage is that people were fired over this.

I'm aware of the tendency to try to focus on what it seems like we can control rather than on actual justice. In this case, I'm so far only seeing outrage about the Easter Egg. I'm not seeing outrage about "They fired people over this???!!! People whose only crime was using Ant??!"

I'm not a fan of what Christmas has mostly turned into these days. I'm ambivalent about leaving a remark that could easily be wildly misinterpreted as supporting and encouraging cultural insensitivity by Americans/Westerners.

Anyway, I may seriously regret commenting at all. This seems like a can of worms for various reasons. I just dislike the idea of standing idly by while the developers get lambasted, as if they should have known ahead of time that people would be fired over this because of stuff happening in China. I'm also leery of the possibility that the other answer will be to ignore the seriousness of the situation for some people and act like "not my problem."

So there's my 2 cents.

Imagine working for the state. Your non technical boss comes in and asked why you have a holiday theme on the page when policy is to not show celebration on state websites.

He asks you why you put this code in. You tell him you didn’t write the code. He asks who did. You tell him it’s open source, many people contributed and someone put this in without telling anyone. At this point, you’ve lost all credibility to you non technical boss and all of his non technical execs. You put a strangers code on the state run website, without reviewing it and without knowing what’s in it.

If your boss can't sell which things are part of their output and which are external, they're not doing a very good job. A default Chrome install might have a Christmas-themed homepage, an ISP might MITM HTTP traffic with a greeting, an OS/hardware manufacturer/ad network/other 3rd-party might do something...

The days where the state run website only runs code written by the guy in the department who knows how to spell HTML are long gone. Everyone is working on a very small subset of the huge stack of complexity that makes someone else's browser on someone else's device made by someone else on someone else's network linked by someone else's CDN to someone else's datacenter that hold someone else's server hardware which runs someone else's OS which hosts someone else's server stack and uses someone else's framework to show the data you actually wrote. If your non-technical boss can't explain that some parts of this chain aren't under their complete control they're just not being honest.

> If your boss can't sell which things are part of their output and which are external, they're not doing a very good job.

It's been a problem in the past:

https://web.archive.org/web/20060427011138/http://www.centos...

> Thu, 23 Mar 2006 00:52:58 +0000 (Wed, 18:52 CST)

> Jerry A. Taylor submitted the following Information:

> Email xxxxxxx

> Company City of Tuttle

> Location Oklahoma

> Comments

> Who gave you permission to invade my website and block me and anyone else from accessing it???

> Please remove your software immediately before I report it to government officials!!

> I am the City Manager of Tuttle, Oklahoma.

And the response:

> From: Johnny Hughes

> To: Jerry A. Taylor

> Subject: Re: www.centos.org - Contact Us Form

> Date: Wed, 22 Mar 2006 18:59:18 -0600

> I feel sorry for your city.

> CentOS is an operating system. It is probably installed on the computer

> that runs your website.

> We hope you are happy with it, since we produced it for free and you are

> able to use it without paying us ... and are even threatening to have us

> arrested for providing to you free of charge.

> Please contact someone who does IT for you and show them the page so

> that they can configure your apache webserver correctly.

> Thanks,

> Johnny Hughes,

> CentOS 4 Lead Developer

Figured out what happened yet, Encyclopedia Brown?

Yep: The city of Tuttle, OK, had a misconfigured/unconfigured site which was showing the CentOS default page. The City Manager of that August Berg thought "We've been hacked! We've been hacked by hackers with a bland corporate logo who left contact information! I MUST THREATEN THEM USING EMAIL!"

Click the link. It gets stupider.

My point is, you have to take the... uh... "violently ignorant" into account whenever you design things which can be public-facing.

”If your boss can't sell which things are part of their output and which are external”

“Don’t blame my personnel. They didn’t write that; they just chose to run random code downloaded from the internet” isn’t an optimal reply to “why is our web site broken?”.

In China, I can see this go towards a “only use packaged approved by the government”, where such packages are signed, and the government knows who to put in prison when a package contains such a surprise.

This sounds like a gross over-generalisation of your personal experience. Why would those days be "long gone" in a Chinese state-run agency?

Because the Chinese state run website is accessible from, say, an Apple iPhone running iOS/Safari. It is probably intended to be accessible from ISPs outside the country. It is developed on Windows computers and hosted on Linux servers, which probably run a server stack that's a combination of open-source modules and proprietary glue.

Granted, your Chinese state-run website is probably more control-oriented than a hypothetical SV startup which consists entirely of connecting VC money to external microservices with a little copy added in by machine-parsing a slide deck...

"If your boss can't sell which things are part of their output and which are external, they're not doing a very good job."

The world is full of bosses who aren't good at their job. And bosses who are flat-out ignorant, capricious or lazy. None of these factors will protect the technical person mentioned in the grandparent post.

And you'd be shocked at how many state/federal resources have a single point of failure. Add to that small-to-medium sized companies, and suddenly there could be a lot of people taking (possibly irrational levels of) heat from their bosses.

> If your boss can't sell which things are part of their output and which are external, they're not doing a very good job.

But it's not -their- job to do that. Their job is to manage resources etc. for the devs.

I worked for a corporation for a few years. We had strict rules about not downloading anything to our work computer because they did an internal study and found that 98% of all malware came from employees downloading music apps and that sort of thing. So they forbade it all wholesale. Breaking that rule could get you in trouble, up to and including being fired for it.

So, sure, it's possible that in some cases people were fired because this Easter Egg outed them for violating some rule. And it's entirely possible that some of the cases of "I got fired for this" are that kind of scenario.

But if this is not against the rules and the firing is actually because it is Christmas-themed, that's a different situation.

I'm also aware that it could be a cultural thing where someone has to be the fall guy and heads will roll to prove our innocence. I see that as deeply problematic. If you didn't forbid open source, firing someone for using it as the "first offense" penalty is not a good policy.

I realize we likely have very little information at this point in time about the details behind the firings and it is the norm for early reports to be wildly misconstrued, which is one of the reasons saying anything at all here is potentially something I will regret.

It's very different if you're a developer. Developers are almost always local admins, and unless you work for NASA, you're always going to be using 3rd party code. You simply cannot write everything (or even a fraction of) the things you need for basic applications in the current space. This isn't the 80s and we're not on Commodore64 machines and punch card systems.

Actually this 'very different' aspect is a broken piece of modern development - its a bug, not a feature. There are methodologies which could have caught this entire stack issue and nobody would have lost their jobs - its just that the ethics of "developer who has control over everything else, or else" versus that of "proper operations and support engineering management (i.e. fire burns upwards..)" are out of skew.

We've been putting up with live fixes and direct "developer"-"production system" style methodologies for a long time; only ethically. It just happens to be 'accepted practice' to fold some gargantuan code-base into ones own environment, without a line-for-line proper review. "Its impossible", say the bean counters. "Who would pay for that?"

Technically there is no good reason for the easter egg to have occurred, had someone done a proper code-review, observed full test reports, respected code-coverage rules and principles, and so on.

The easter egg proved that someone wasn't doing their job.

Really? So some team should have inspected the entire Linux or FreeBSD kernel they were running one? Someone should have gone line-by-line through nginx? What about all the shell commands used? They should have had people go line by line through bash, cat and all the other tools needed in the background to setup and environment, all their libraries, all their dependencies?

That's not even getting to a project. Have you looked at a projects node_modules, or the maven/jar dependences pulled in? Even for small projects, it grows to a very non-trivial amount very quickly. Inspect ever single jar and dependency?

Should people write their own web servers instead? Their own frameworks? Their own operating systems?

At some point you have to trust someone enough to actually get your work done. If an OSS project breaks that trust (like we've seen with some node modules) you stop using them, but inspecting every last dependency is often impractical.

> all their libraries, all their dependencies?

Well, you see ... I happen to think that you can use a lot of tools to cover your ass, and .. the fact that this one slipped in is as much a comment on the crud that is promulgating the wild and woolly Node/JS ecosystem as it is anything else. In point of fact, this kind of bollocks is why I eschew Node/JS and use other things [1], instead.

I do believe there are tools and ecosystems which make this sort of thing less likely. I can't recall a Linux easter egg .. nor a Golang one ..

>Should people write their own web servers instead? Their own frameworks? Their own operating systems?

One should at least, audit. As much as possible. It doesn't take much for a competent dev to 'grep -ir "easter egg $CODEBLASE' or, whatever .. not that its an expectation.

But yeah, if you have to have government-level 5-nines on all services, then I would say - fair play. The responsibility for an audit of such things should definitely have been in the requirements. I've seen such expectations for lesser projects, personally, where .. indeed .. code audit and ownership were tightly .. and properly .. managed.

[1] - I don't know for sure, but I think its harder to slip in such an easter egg on a production golang system. I guess I'll tune into that if/when it happens/has happened..

You put a strangers code on the...website, without reviewing it and without knowing what’s in it.

There is not a single person in the entire history of web development that this sentence does not apply to.

Bullshit. IBM at least takes dependencies very seriously and while the system is not infallible there are persons whose role is to review what the heck people are including in the software, making sure there aren't hidden backdoor or potential licensing issues.

IBM contributes to the Linux kernel, so it might vet that one component, but does it vet every single component in an actual, functional distribution? That would be absurd.

No way they do, I included go packages and npm modules that were not vetted by others or myself. They do have a license review, with many licenses being pre-approved.

They probably all would have gotten fired too.

Reply on news.ycombinator.com