Hacker News Reader

Comment by oneeyedpigeon | Hacker News Reader

> Early June 8, a customer pushed a valid configuration change that included the specific circumstances that triggered the bug, which caused 85% of our network to return errors.

Is it necessary to refer to "a customer" at all in this statement? What would be problematic if the above were rewritten as something like:

> Early June 8, a configuration change triggered a bug in our software, which caused 85% of our network to return errors.

The advantage is that you wouldn't get ignorant reporting that "one customer took down the internet". I'm not sure there are disadvantages that net outweigh that.

View on HN

Yes, because it is explaining that it was a valid *customer* configuration, which is a separate set of concerns from, say, infrastructure config.

The important adjective "valid" means it was completely normal/expected input and thus not the fault of the customer.

It's perfectly clear you've come at this with a pre-determined agenda of "I bet fastly, like most other public statements after corporate booboos I've seen, will try to shrug this one off as someone else's fault" after reading the BBCs title and haven't bothered to read it at all until now.

Not necessarily valid. Could have been a bad entry that passed validation when it shouldn’t have, which would still not be the customer’s fault.

> We experienced a global outage due to an undiscovered software bug that surfaced on June 8 when it was triggered by a valid customer configuration change.

Verbatim from Fastly: https://www.fastly.com/blog/summary-of-june-8-outage

> Is it necessary to refer to "a customer" at all in this statement? What would be problematic if the above were rewritten as something like:

That's literally what happened. They even say it was a valid configuration change, it's very blameless.

Saying "a configuration change" loses critical context. I would have assumed that this was in some sort of deployment update, not something that a customer could trigger. Why would you want less information here?

OK, I'm replying to your comment since it's the least aggressive — thanks for that!

I'll fully retract my statement. This is 100% the BBC's fault, 0% Fastly's.

Can I make one small suggestion that might help to prevent this kind of misleading reporting in future, though? What if Fastly produce the detailed statement they have, with as much accurate technical detail as possible AND a more general public-facing statement that organisations such as the BBC can use for reporting, that doesn't include such detailed information that can easily be misconstrued?

Most of the replies to yours haven’t been aggressive. Ironically it’s your comments that have come across the worst by using terms like “aggressive”, “blame” and “fault” in the first place. Calling other people’s comments aggressive is pretty unfair. One might even say hypocritical.

I hate being part of a dogpile, so yeah sorry about that, I just open up things to reply to, and then come back later and write it up just to find that I'm one of 10 people saying the same shit.

edit: FWIW I had a very negative initial reaction to the headline as well.

Not at all — I understand.

My apologies for any hostility on my part.

No worries. I probably didn't take it very well because my intentions were genuine and I really wasn't trying to level anything beyond the very mildest criticism towards Fastly. I recognise, however, that even that was misplaced — I think the BBC headline just got me too worked up!

In what way is the BBC at fault for this? Their title is objectively true. A _valid_ configuration setting that was used by a customer _did_ cause fastly to have an outage.

It's not limited to one specific customer (i.e this customer isn't the only customer who could have caused the issue, presumably), but it _was_ something the customer (legitimately) did. It wasn't a server outage. It wasn't a fire. It wasn't a cut cable.

"a customer quite legitimately changing their settings (BBC: one fastly customer) had exposed a bug (BBC: triggered internet meltdown) in a software update issued to customers (fastly admitting, when combined with 'legitimately', that fastly are at fault) in mid-May".

People love to hate main stream media

Not me — I adore the BBC. I've always paid my licence fee gladly, and I've been waxing lyrical about the latest BBC drama on Twitter just this very hour. On this issue, I believe they've made a mistake.

Whatever happened to nuanced opinion, where you can see good and bad in the same entity? Why do some people insist so strongly on absolutes?

How on earth is the headline making a mistake?

Here's some excerpts:

Fastly, the cloud-computing company responsible for the issues, said the bug had been triggered when one of its customers had changed their settings.

Fastly senior engineering executive Nick Rockwell said: "This outage was broad and severe - and we're truly sorry for the impact to our customers and everyone who relies on them."

But a customer quite legitimately changing their settings had exposed a bug in a software update issued to customers in mid-May, causing "85% of our network to return errors", it said.

The headline accurately portrays the story given the limit on headlines.

The somewhat awkward "a customer pushed a _valid_ configuration" is Fastly making sure they aren't pushing any blame onto the customer.

There is no customer blaming here. None at all.

> Is it necessary to refer to "a customer" at all in this statement?

That’s how autopsies work. You describe the cause and resolution. The cause was a bug in the customers control panel.

They’re not trying to absolve themselves of responsibility.

The wording "a configuration change triggered a bug" in this context sounds (to me) like it was a configuration change made by Fastly to something on their backend.

The wording which was actually used makes it clear that that was not the case.

Reply on news.ycombinator.com