Hacker News Reader

Comment by kneebonian | Hacker News Reader

> Of course the personnel in department Z might review it but ignore it

That's the problem right there. One of my clients had a large IT organization of over 1000+ employees, with strict change control rules, and procedures that were tracked in SN. Every time there was a change management meeting everyone who could possibly be effected would get an email from Service Now notifying them of the upcoming changes.

Pretty much engineer ignored those emails, because there was so much going on in the org you'd get dozens of emails in a week and most of them you'd only be tangentially effected by, meanwhile you had your work to do.

So the problem isn't getting the notifications out it's getting people to pay attention to them.

View on HN

It sounds like the actual problem is getting too many meaningless notifications, inadvertently training people to ignore everything

In an ideal situation, you'd get all "planned maintenance" emails for things you care about and no emails for the rest of them.

That (probably) means that the system for dealing with planning maintenances (well, usually, "approving them") needs to have a sufficiently good understanding of what humans care about what changes.

At a previous job, the planned change tracking system was REALLY good at tracking what specific compute facility was going to be impacted by any specific change taking place in that facility. And had a really good way of allowing you to filter for "only places I have stuff running" (and I think, even some breakdown of general change types as well).

It was, however, not easy to get notification of "there will be maintenance on submarine cable C, taking it off-line for 4 hours" or "there will be maintenance at cable station CS, taking cables C1, C2, and C3 down for 3h". And as one of the things "we" (the team i worked in then) was doing was world-wide low latency replication of data, we did actually care that cable C was going to be down. But, the only way we could find out was "read all upcoming changes" and stick them in the team calendar.

Was it good? Eh, it worked. Was it the best process I've seen? Probably ,yes.

I don't see how that's a problem for the organization.

Individual preferences do vary, one ignores 90%, another 95%, another 100%. And the one who's ignoring 100% of them will likely eventually make a mistake that otherwise wouldn't have happened.

But it will be fairly straightforward to resolve, after all there's an extensive paper trail as the chain of custody seems clear. Assuming the "change management meeting" emails were the approved means of communication.

> I don't see how that's a problem for the organization.

IMO, one of the lessons that came out of Chernobyl is that it absolutely is a problem for the organization. Exposing people to too many "alarms" that are constantly going off will cause people to start ignoring them.

Part of good design is figuring out which things are truly important, and how to communicate that to the people who are supposed to be paying attention.

The analogy seems not to apply?

The emails mentioned by the parent don't sound like alarms. Because an alarm is usually for 'drop everything and focus on this' situations.

The equivalent in email terms would be a receiving an email with a subject in ALL CAPS bolded and underlined.

Or in general intra-company communication terms, a phone call from your boss without any pleasantries and a serious voice.

Alarm or not doesn't really matter. If a person is receiving a signal that does not affect them most of the time they WILL start to ignore that signal. Many will attempt to combat this with policies and consequences, "Make sure your reading these e-mails, or else!" but it's a fruitless endeavor. Humans will human. Better to recognize that and build your systems around it.

So what?

If someone makes the wrong decisions because they start ignoring signals then don't promote them or give them important coordinating responsibilities. Those who are capable of filtering out a larger fraction of noise do exist.

Of course there will always be folks whose preference is to read near 0% of their emails, but that doesn't imply organizations must be designed around them.

Finding someone to blame doesn't matter if the company goes bust.

It's not about playing the blame-game, it's simply to make sure the personnel in responsible positions are those who can handle it, in a verifiable on-the-record way.

> If someone makes the wrong decisions because they start ignoring signals then don't promote them or give them important coordinating responsibilities. Those who are capable of filtering out a larger fraction of noise do exist.

This is simply wishful thinking. Outliers certainly exist, but the idea that there are sufficient number of them that you can just ignore human nature is a path to disaster. You'd have to somehow accurately measure not just who is opening these noisey e-mails, but what they are retaining from them, and measure it over a large period of time, knowing that the vast majority or going to fail. It's far cheaper and more reliable to fix your noisey system than to try to outwit human nature.

> It's far cheaper and more reliable to fix your noisey system than to try to outwit human nature.

Can you describe this 'cheaper and more reliable fix'?

Yes. Make sure you are sending people notifications that are relevant to them, instead of sending all your employees the firehose and relying on them to pay close attention on the off chance something they need to know actually slips in.

Can you describe who would be responsible for deciding what to send and what is relevant?

Sure! The people administering the system define these types of rules according to the shape of your organization. It may surprise you to know this is a core feature of most change management software. It's not like "target your notifications to a specific group of users" is a novel idea.

How would you envision these 'types of rules according to the shape of your organization' be set in a way that doesn't cause endless politicking and horse-trading?

By assigning them like rational human beings?

It's really weird that you think you can't decide who is responsible for dealing with a type of notification ahead of time without "endless politicking and horse-trading", but think that blasting everyone with every notification, then attempting to sort out responsibility after something has gone wrong will somehow not cause "endless politicking".

Again, this is something many businesses do already. They don't blast the whole company when a bank account balance is low, when a server's disk is full, etc, notifications are targeted to appropriate groups. The specifics about who gets what is going to vary based on your organization. I can't give you hard and fast rules that will work for every organization, but that doesn't mean it's somehow impossible or not worth doing.

> By assigning them like rational human beings?

It's hard to tell if your joking.

The reason for implementing restrictive organizational procedures in the first place IS because nobody behaves like a 'rational human being' in such an environment for any significant duration.

You appear to not have experienced intra-corporate spam.

When __everything__ is highly important and #urgent#, nothing is important and urgent.

'Everything' may be labelled or described as 'highly important and #urgent#' but they may not correspond to reality.

In a sufficiently large and complex organization there will always be some percentage, neither 0% nor 100%, of actually 'highly important and #urgent#' hidden among the pretenders.

The point of any such tracking system is to discover, via positive or negative selection, what is actual instead of what various flawed humans pretend.

It sounds like what you're suggesting is that, so long as you know who to blame for the problem, it doesn't really matter how bad the problem is when it hits you? Even if the company goes insolvent because of the problem, if you've got someone to point to and say "their fault", it's not a problem for the organisation?

That... doesn't sound like a great approach to me.

It does presume there is someone, or some group, keeping track of the tally on at least a weekly basis.

If no one is keeping track of who is doing well and who to replace, then fixing that first is more important.

Well, there's supposed to be a group keeping track of the alerts about anyone who is doing badly, but one of them is ignoring 90% of their alerts, another 95%, and another 100%. It's OK though, because the system is built to handle that kind of problem - by keeping a paper trail of who is to blame. Marvellous.

:-)

If the organization is so large that it needs even the tracking group to have its own internal hierarchy then I'm afraid my wisdom is lacking.

Reply on news.ycombinator.com