More than five whys and “layer eight” problems
rachelbythebay.com
rachelbythebay.com
I've worked with so many truly incompetent and stupid teams and organizations, that have millions to billions of dollars. Sometimes I wonder if I've just hallucinated all the books and training I've taken that explain the basic business and industry concepts most people should know. And I wonder how it is an IC could seem to know how things should be working while seemingly nobody else does.
Then I go back to my 1/30th of the widget and wait for lunch time.
I have probably written more code in the last 6 months than the preceding 5 years. It's not because I suddenly got smarter or I am some 10x engineer, it's because all the BS got out of the way.
No agile, no jira, no backlog grooming, no story points, no standups, no sprints, no product owner, no stakeholder management, no churn. No "two masters" or principal-agent problems.
Client/boss knows what they want, and so thats what I work on. It's clear whats important, so that's where I spend my time. He trusts my advice and feedback because he doesn't have 10 other senior engineers whispering conflicting BS in his ear. He is clear on how robust a solution they want, and if it's worth moving onto a new idea or wrapping up the current one.
So, freed from all the other problems, I just build.
Just building by myself has an upper limit. To be fair, that upper limit is roughly 20% of what that whole 200 person org does, but still. I keep imagining what we could accomplish if we reached even 10% of the theoretical limit.
One path may get tiring after a few cycles, and you take the other path for a few.. rinse & repeat.
Yes, it is. Not sure what you include in "books and training" but the actual hard stuff is the complexity inherent to each individual that you want to organize. Each one of them has moods, goals, desires and bad days. And the number of parameters is huge and they're all different for each individual. And it takes just one member of the group to be sufficiently aligned in another vector to make it all fall apart.
And I am just talking about a team or small department. Figure this at state level and you get why politicians actually don't get anything done.
Not really. It's a handful of common-sense techniques around communication, architecture, and getting out of people's way. There's 3-5 books that compress most of the necessary knowledge into straightforwardly readable form.
Of course, most people haven't read them, let alone applied them. Instead, as an industry, we've converged on having engineering companies led by non-engineers, "scrum masters" galore, and desperate cargo-culting of OKRs instead of having real objectives and real key results. "Top" teams at FAANG companies are routinely having 1980s-style conversations about "staffing projects" while they reassure each other that they don't need to prioritize because "everything is important." (Ask me how I know...)
It's all self-inflicted. I don't know about your extrapolation to broader politics, but back here in the office? The simple task of organizing 5, 10, 50, maybe even up to 100 people? That part is perfectly doable without making any major mistakes. We just don't want to do it.
At minimum, though, I wish we'd stop sitting around and saying it's too hard. Here's Ben Horowitz, writing in 2011 (twelve years ago!) [1] about it:
> If you manage a team of 10 people, it’s quite possible to do so with very few mistakes or bad behaviors. If you manage an organization of 1,000 people it is quite impossible.
If we as an industry could hold the bar higher on running companies/teams when they were this small size, personally I think a lot of things would naturally sort themselves out at the 1,000+ person size. Besides, change begins at home — even just improving the running of your local team would go a long way. Then, even if the executives are flailing around with their latest harebrained take on why they don't need to write rigorous OKRs because they need to run off on yet another vacation, at least your team is good.
Leading a group of a dozen engineers is a perfectly straightforward and learnable skill. And it doesn't "take just one member... to make it all fall apart," because you can just fire that one member. I'm not claiming all this stuff is trivial, but I wish we'd stop making it sound like some sort of mind-blowing skill of living on the knife's edge and wielding secret, arcane leadership knowledge.
(Sorry for the morning rant. I know you meant well, and you're also right to empathize with each person's individual needs. It's certainly a difficult job... it's just not impossibly difficult.)
[1]: https://a16z.com/2011/03/31/whats-the-most-difficult-ceo-ski...
EDIT: I posted my personal list of 3-5 books below. I'd love to hear others' lists, too.
- Peter Drucker - The Effective Executive
- Andy Grove - High Output Management
- Fred Brooks - The Mythical Man-Month
- Marty Cagan - Inspired
"Workplace Management" - Taiichi Ohno
"Lean Six Sigma" - Jeffrey Ries
But aside from these "management philosophies", there's also standard industry/role practices that companies don't require their employees to implement, and when they do it's only enough to say they technically did it. Often the employees don't even know the standards exist. It's embarrassing.
Agreed; my example would be all the basic things in "High Output Management" — especially the performance management pieces.
What'd you have in mind?
- Dale Carnegie - How to win friends and influence people
- Jocko Willink - The Dichotomy of Leadership
I would also recommend The Goal by Eliyahu M Goldratt as an excellent primer on the Theory of Constraints as an effective management philosophy.
Most people recommend The 7 Habits of Highly Effective People and other books in that ilk, but I've found far more value in these three books alone than every other management theory book I've read, combined.
And if you go and read some old management cybernetics you see all that stuff back in the fucking 60s, 80 years ago. I think you absolutely hit the nail on the head tho
- Management 3.0: Leading Agile Developers, Developing Agile Leaders (pushes a team-first approach that I think supports and reinforces the core concepts of agile )
Just recently I came to a realization, people and organisations exist in a competency / incompetency matrix.
The competent orgs with incompetent people are your blue chips and "traditional" companies. They ship stuff, have customers, grow along with the overall market and economy and are abble to survive most non-existential threats and down-turns and even existential ones from time to time. All, or most, the competency is institutional, recorded in implicit knowledge held by people and various tools and processes. Unless those companies loose that knowledge for some reason, things are good.
Competent orgs with competent people are incredible places, e.g. Amazon. They grow like crazy, rarely screw up and develop. They ship and have happy customers, they innovate. These places are hard to come by, harder to create and even harder to maintain.
Then you have incompetent orgs with competent people. By definition start-ups, those only reach competency when they serve, sucessfully and not depending on outside funding, markets and customers for a while. Doing things right, those orgs can create orgabisational competency based on the people competency.
And then there are incompetent orgs with incompetent people. Places were you think, as you said, you haluzionate all your training and experience. The only thing standing between a start-up (incompetent org and competent people) and such a nightmare is leadership. The thing that allows everyone to pretend to not be obe of those doomed places is endless cash. Sooner or later, those places come crushing down.
I shared room with people probably a bit less skilled at working than min wage clerks I worked with a few years ago. But they got a Java cert so they think spending hours in meeting is work, and efficient.
We should have mandatory non IT work to make people see what it is to work lean and fast.
And all of them have deemed you a suitable fit for their incompetent and stupid team or organization.
As someone who works as an educator at an art school where chaos is the rule, I must say if your company is less organized than a spontaneously formed group of twenty year old crazy-chaotic (and probably drunk) art students, you should probably just close down operations and change your career.
And yes, I have seen such organizations, companies and "professionals".
Why? It brings in boatloads of money.
I see your point though. Software has grown so big and detached from the problem that people don't get proper feedback about what they delivered and that makes for a bad quality. In niches it can be better that is why I started that way.
it's motivational in some ways seeing how dysfunctional even successful companies are
got a list of them for reference ?
- Don't build product before talking to prospective customers. Don't build your products in a vacuum.
- Pay your suppliers on time.
- Take good notes. You'll need them later when a CEO is breathing down your neck about why his darling (yet useless) feature was removed. It's hard to remember context around decisions that were made X months/years down the road.
If I invest that £100 instead in another part of the business, that returns £150 a year on average. After two years, that's already £225 - enough to pay for the crisis and still have made £25 profit. If I can get a third year without the server breaking, I'm closer to £140 profit, and after four years I can already afford to have two crises off the back of my initial £100 crisis-avoidance money.
Obviously this is a very simple example, and there are lots of examples where this wouldn't work out. If my rate of return was only 10%, then it would take eight years before I'd have made back enough money to pay for the crisis situation. And sometimes gambles don't pay off and the crisis happens sooner than you expect, and other times people are just consistently bad at estimating these sorts of odds. But the point is that, even if it's cheaper to do something now then later, if you can find something more useful for that money to be doing right now, it might still be more profitable to take the expensive option.
(That said, there's also obviously the human cost - if your team is constantly operating in crisis mode because everything's been used until the last possible moment, then it's unlikely to be an effective team for very long. But then that's kind of the point here: this is all a matter of tradeoffs, and trying to predict the future as best as possible with limited data.)
Realize it has been written almost 50 years ago.
But other than that solid point.
Even without knowing the principles I would expect people to stop talking if they don’t have knowledge - but most annoying thing is when people don’t know and are trying to help genuinely and they are convinced they are contributing.
I gained two key insights from experiencing this several times.
First, the really useful diagnosis often transcend realms. Eg, what may have occurred as a technical issue was really the result of a process issue, which may have been a result of a hiring issue, which may have stemmed from the personality of the manager who made the hire.
Second, the goal is to necessarily to find the "one true cause" but to drive some improvement. For example, if something was caused by a confluence of factors, if you dove in and improved just one of the factors, you've already made the situation better. You may still chose to fix just the tech or process, but the insight into the deeper causes helped shape our understanding through time.
"The Second Law of Consulting: No matter how it looks at first, it's always a people problem."
- Gerald Weinberg - The Secrets of Consulting
Maybe a psychology degree would be a better background to pull engineering managers from.
Believable relationships are scarce in the workplace.
One aspect that I like to emphasise is the systemic or second order aspect. Yes, doing a retro/rca/post mortem/coe to prevent reoccurance of the same problem is good. Better is to prevent similar problems, or even classes of the problem, across the system in the future. A quick litmus test is "Do the lessons learned and action items apply to the observed problem, or do they apply to similar problems."
A solution that improves information flow solves problems you don't even know existed, yet! This is one way to deal with unknown unknowns. (Or black swans, depending on whose terminology you prefer.)
1. Dig into the root causes. Five Whys is at least helpful in giving the idea that the root causes may be several layers away; you should definitely go past 1 or 2. Untrained humans find it very cognitively tempting to stop at exactly 1; this must be overcome and the Five Whys meme is a good entry point for that.
2. Don't be stuck on Five Whys literally, as essentially a linked list of exactly length five. Causes can be trees. (I mean, technically nothing stops them from being full graphs, at least from a high-level view that ignores temporal dependencies, but generally you need to tree-ify them for this discussion or you'll literally loop around the graph in your postmortem and analysis steps just like a computer would!) A 5-element linked list is merely a particular shape it can have. Management encouraging "Five Whys"-like thinking is a good thing; management mandating that an incident come with a Five Whys analysis with exactly five whys in a linked list would be missing the point in that oh-so-classic management methodology way.
3. Once you have your root cause tree, you examine it with the same engineering/business cost/benefit eyes as everything else. "The entire company's organization is inimical to this sort of quality" may be true, but extremely expensive to fix on its own terms. Even more so when you look up and consider that the company may in fact have other good reasons to be organized that way; there is no perfect organization that solves all problems optimally, and it may be a net loss to even consider re-orging the company to solve one particular thing that in the global context is an annoyance, if it would in the process break the organization that is producing actual value. But you may find a process tweak in some other layer that will mitigate 80% of the problem for hardly more than the cost of a meeting. It is perfectly valid to come up with a portfolio of solutions; one pattern I end up seeing a lot is the cheap short-term bandaid and the longer term fix. Cynicism aside, yes we often do end up with the long term fixes getting put into place... the key there is that they are usually themselves a portfolio of other fixes we needed that long-term fix for, not just this one issue that we've put a bandaid over.
I'm yet to ever participate in an analysis like this that didn't produce some good actionable item. Often at least one of them is indeed quite cheap, even if fixing the root causes is expensive.
5 Whys are used a lot of time to shift blame away from the team where the issue originated, or someone does the 5 whys dogmatically and ends in a non-actionable final why.
One of the things I teach is "5" is a suggestion, not a requirement. Trees are great. Each root node in the tree MUST be actionable. Asking people to do better next time isn't actionable. Blame isn't the goal of the document.
5 whys is a tool in root cause analysis. The point of 5 whys is to identify links where "failure would not have happened if" and not act on those.
Blame is a tool that helps dig deeper by aligning incentives. As a moderator you want to find those ifs and blame helps point the finger at perceived causes. The hard part is triage done on blame shifting to assess whether that is actually a cause or just an excuse.
I didn't work for Bridgewater but I spent some time working at Bridgewater (for a vendor).
Edit to add: Bridgewater's diagnosis process typically goes waaaaaaaay deeper than your standard 5-whys. Like the difference between the doctor just taking your blood pressure and the full physical where they put on the rubber gloves.
[1] Like literally. People will quote principles to you by number if they think you have violated some of them.
It's kinda the opposite of "soul sucking"
There are bad examples also, especially in cash-rich companies, who tend to be infested by parasite managers. Those are not interested in org optimizations.
Do you know of some publicly available material on this? I’d love to hear about their approach.
https://www.principles.com/principles/85cda87a-6491-46bd-9cf...
In big corporations many problems occur because of process/data siloing and impedance mismatch in the inter-departmental communication. Everyone is building their own, no organic intensives to share and cooperate.
/s
I've successfully pushed through organizational changes at least a few times from the output of the 5 Whys process too. If the COE/postmortem revolves around a critical issue or has gravitas, it's easier to put your full weight behind it.
To be honest, my personal hit rate on these is probably about right. If you insist on it every time, you're probably going to burn out on it, personally or organizationally. Sometimes it's OK to try to address a symptom, if the medicine is cheap, easy to administer, and has little to no side effects. And in tech, the symptoms are often inherently layered -- you can often address entire classes of symptoms in reasonable ways, /even if/ that solution is itself working around a deeper organizational shortcoming / root cause.
Spend the friction tokens where it matters most.
In the "car won't start" example of the original article, the motor pool mechanic needs to stop the whys at the broken alternator belt, because that is something they can and should fix; the motor pool manager needs to go on until reaching the practice/policy for how often preventive maintenance is done, and the top management (if the consequences are severe enough to warrant attention) need to go on until the "why do we have poor car maintenance" questions reach things like mid-management forcing people to ignore the maintenance schedules, or staffing policy, budget for whole departments, and/or the choice to outsource/insource that function.
But going on much further than your own level is not going to be productive - you can make an actionable suggestion to your boss if some root cause isn't fixable at your level but is fixable there, and that's about it, at least in my experience.
That's a great quote that I shall use in the future.
“ A manager truly appreciates original mathematical ideas. The [same] manager requests to rid the code of crash-causing bugs, because customers resent crashes. The most confident people ignore him and spend time coming up with original math. The less confident people spend time chasing bugs, are upset by the lack of recognition, and eventually leave for greener pastures. At any given moment, the code base is ridden by crash-causing bugs.
“ A manager enjoys "software architecture", design patterns, and language lawyer type of knowledge. The [same] manager requests to cooperate better with neighboring teams who are upset by missing functionality in the beautifully architected software. People will tend to keep designing more patterns into the program.”
It suggests a unique and amusing approach to hiring management: executives should find people who truly value the goals of the executive, hire them as managers, and (perhaps via intentional lack of oversight!) give them leeway to let their subconscious values impact their work, thus making their minds legible to mind-reading employees.
"Hire people who share your values/goals and let them get on with things" is hardly a novel insight. The trouble is that aligning them is hard.
Right, because it’s not the same thing. If you hear “hire people who share your values/goals and let them get on with things” and implement it, you’ll likely end up focusing on hiring developers/engineers and giving them resources, rather than hiring managers and giving them fiefdoms to run according to their whims. You’ll also likely end up hiring people whose true values match your true values and whose stated goals match your stated goals, but this argues that you want people whose true values match your stated goals, and you should ignore their stated goals. Despite sounding similar I think it is really quite different.
(Arguably, this is WHY we find “alignment” so hard: we are constantly aligning their values with our values and their goals with our goals, and we are even achieving that alignment some of the time, and yet things still don’t work - instead we should be trying to align their values with our goals, because this is the kind of alignment that produces results)
Only if your goal is your stated goals and not your true values, which by definition it isn't.
The concrete examples above of valuing fancy math or design patterns over product quality helped clear it up.
To illustrate consider a drunk pedestrian jaywalking and being hit by a speeding driver who was texting while driving. Who caused the accident? They both did, together. Or maybe one of them did and the other didn’t. Or maybe the stars aligned just right and both would have been fine if they hadn’t been doing it at the exact same time?
I can’t help but feel like anybody who has found the “root cause” for any problem, their answer will reflect their biases far more than will reflect true causality. If you can come up with a single answer for a complex causality problem, something hints to me that you would probably point to the same “cause”for other problems too.
> dramatic oversimplification
> single answer
If someone pitched it to you that way, I agree that's a dramatic oversimplification -- not of causality, but of what's meant by the "Five Why's."
"Five Why's" is a tool, a reminder to separate the symptoms from <waving hands> whatever complex system dynamics that caused them. Nothing about that analysis has to be linear nor yield a single answer.
The key lesson is to never solve a problem on the same level it was created -- i.e., the symptoms. Take the time to think through the causality. If you can go five levels up/away from the symptoms and still find an actionable mitigation for a whole class of problems, great! More likely, you go up one level and at least you won't experience the exact same symptom twice.
I just recently did a 5 whys session on some production data loss. We ended up with a branching tree that had 22 items and was up to 6 items deep. In the past I've ended up with bigger and deeper ones. To me, the "5" is just a reminder to really dig.
For me it's also a practice that's most valuable when used frequently. As you see similar things crop up in different sessions, that's a great sign to dig deeper. E.g., one of the leaf nodes in our recent retro was "no official process". That's deep enough for us; we'll create an agreed process for this particular thing and put it in the wiki. But if I see that a couple more times, I'll definitely drive the cause analysis deeper.
My personal experience has been anyone taking the "5" part literally is a cargo cult 5 why'er and they will not produce anything of value from the exercise.
An example of how to illustrate the wrong behavior is -- for software devs -- something like a null pointer dereference. I've seen code reviews where the suggested fix is simply to guard the dereference to check if the pointer is non-null. Sure, sometimes that actually is the right fix. But a lot of time - maybe even most of the time - you need to to take a step back and ask yourself why it was null, whether it should have been and what else must change if it shouldn't have.
Then again, NULL was a $1B mistake anyway, so why not...
reWriTE It iN RUST, bABy!
The intention of the five whys is to find breakdowns at each level...not just the last one, and mitigate each.
Ultimately 5 why's is an arbitrary guideline that balances trying to keep you from being complacent in your analysis, with the need to actually like... stop and make a decision.
--
Therefore, Your Honor, my client is innocent. Why has this tragic accident occurred? I shall tell you to look no further than the root cause...
The Big Bang.
That's basically the point of the 5 whys approach. You work up the tree/into the graph of contributing factors based on the specific problem and you end up solving lots of problems. If you don't go back far enough, you're likely just putting a band-aid on the problem and some other variation of it will waste more time again. You get "5 whys" back and now you can really start figuring out what went wrong and how to fix/prevent/mitigate/avoid/etc. it for next time.
I don’t have any problem with the idea that you should dig deeper for any form of causal reasoning. My problem is with the inherent linear flow and singular root cause reasoning of the 5-whys methodology. It is just too easy to lead you back to the thing that you are personally biased to think anyway. If you want to solve actual problems, you more than likely have to solve multiple problems that contribute to a complex system than to solve any singular problem.
If you can work your way back from a smelly kitchen to a change in religion, you ask, "Does anyone who didn't change religions have a smelly kitchen?" Obviously the answer is yes, so flag it as possibly correlated and move on to investigate factors that have a better chance at causality and aren't so easily dismissed. Or maybe you need to go even further back, and there's some link between why your kitchen smells and why you changed religions. It's just an approach, like therapy, not an algorithm.
If you read this thread there are literally multiple interpretations of what the 5-whys actually means, each of which results in a roundabout No True Scotsman argument. Including yours.
I’d prefer to see the 5-why’s as what it actually is: an aphorism about not treating symptoms but rather problems. The aphorism is useful…attempts to define it as some kind of formal tool are a waste of everybody’s time. There are plenty of tools (such as Ishikawa diagrams) out there that represent a better implementation of the aphorism, and 5-whys can be replaced entirely by them.
And in doing so, you likely realize that you came close to disaster a number of times and never realized it.
If each of those 3 whys have 2 answers, you would ask why 6 more times (n=10, depth = 3)
If each of those whys have 3 answers, we would need to ask why a further 18 times! (N=28, depth = 4)
That is leaving out the last depth, but suffice to say there are a lot more than just 5 questions. It seems rare for any system where the cause and effect tree is linear. Hence, 5 whys is meant to be very holistic, not just deep.
For your example, if texting and driving is associated with 80% of pedestrian collisions but a drunk pedestrian is only associated with 5% of such collisions, you can probabilistically influence most potential outcomes by focusing your effort on the texting-while-driving root cause.
It's funny how humans try to simplify things to black and white all too often. Why do we do this? I'm sitting here trying to think why I do it, and I can't figure it out. Something maybe about needing someone to blame.
For example, when I think of the 2008 financial crisis, I always feel this need to find one group to blame, but really there wasn't one group.
It should also be combined with other techniques that elaborate, more properly, on the model of the system in which the problem occurred. There you will find the deeper systemic causes and can start addressing and mitigating the issues. But that will take a lot longer to explore and resolve than the 5 whys answer of "Get a proper tuneup every 5-10k miles".
[0] Ideally fully, but 5 whys is an expedient, non-thorough technique so that's a reach.
No, you have it exactly backwards. Five whys is explicitly depth first search. It was originally employed by Toyota Motors, and the key to actually getting at what the actual causes of issues were. This allowed Japan to avoid entire classes of defect due to their acceptance and implementations of the teachings of W. Edwards Demmings. It never made the leap to the United States, because doing things right is such a foreign concept over here, where management would gasp at the concept of a single worker shutting down the entire factory upon detecting a problem, whereas in Japan, the andon was key to getting issues that would compromise the Quality of the end product addressed.
It's:
I have a problem, Why? Because of this other thing. Why? What causes other thing? The other thing is dependent on yet another thing and that thing. Are those things a problem? If so, why?
Iterate until done.
The "Five" aspect came into play in that after a certain number of "Why's", it was often the case that the fact it took so many Why's to get to the root cause, that the complexity itself constituted it's own type of defect.
I might need to revisit the book I first read of it in, believe it was titled "Elegance" or some such.
EDIT: Found it.
In Pursuit of Elegance: Why the Best Ideas Have Something Missing by Matthew E. May
EDIT2: Nope, false alarm. Drat it, I will find that reference. It's in this library somewhere... In the meantime, forget it; maybe I'm hallucinating it, because clearly I haven't refreshed it in a while, and the old wetware has taken a hell of a beating over the last couple years.
I didn't say it wasn't depth first so I have no idea what you're responding to with this. Can you point out where in my comment I said it wasn't depth first?
I just said it's bounded, because it is. Even the link you've posted states that "why?" is only asked 5 times. But even if you take the 5 as suggestive rather than a strict limit, giving a limit (even a soft one) implies a boundary to the search. If it was meant to be unbounded it would just be called "Whys". 5 pushes you beyond the proximate cause and towards the actual root cause (of course, "root cause" is itself a poor phrase as it implies a singular cause that can be addressed to prevent the problem from reoccurring), but then the root cause could be 10 or 20 or 30 whys deep. So unless you go beyond the point suggested by the name and publications about it then you won't find it.
------------
EDIT: I see you edited your comment after I hit reply and didn't mark it as such, good form.
Anyways, about it being depth first I'd actually disagree. Even if you believe that it should be taken as an unbounded search (which I also disagree with, obviously, because there are better tools than 5 whys for such a major task) it should not be an unbounded depth first search. That would be a form of malpractice.
The more accurate description would be something like iterative deepening. You go down a few layers with something approximating a depth first search, and then you back track and try the other branches. If you go down one rabbit hole arbitrarily deep you may or may not find a causal factor but you will have missed many others along the branches you chose to ignore for the sake of your unbounded depth first search (again, a form of malpractice).
This is one of the major reasons why I chose to make less money working at a smaller company, as it isn't big enough to accommodate these "reality distortion fields".
I keep wanting to read Moral Mazes, but unfortunately I've yet to get to it.
Exactly. Or as I like to put it: With great responsibility (should) come great power.
The animals have bought all the expensive shiny musical instruments, in order to build an orchestra. But nobody knows how to play (and how to play together). So the only thing they can adjust is the seat order arrangement, stings to the left, brass to the right.
One of the companies I worked for had licensed all possible expensive applications, databases, analytical tools. But nobody knew how to make use of those. There were many initiatives, but all efforts were spread thin.
Perhaps in German this would be called Gießkannenprinzip (the water sprinkler principle)
Compliance is all about reducing risk to an acceptable level, but if you care about it, it is also about reducing risk to as close to zero as possible. Compliance generally affects multiple organizations, and that means convincing a bunch of people to do something that doesn't help them accomplish their primary goals.
To be clear, I'm not alleging any wrongdoing whatsoever against any group that I worked with. I'm simply saying that it's hard enough to get people to do the job they accept as their main job; adding 'extra' work is never a rewarding experience - especially as an individual contributor.
It sort of reminds me of this thought experiment that Nassim Nicholas Taleb mentions in one of his books: What if a bureaucrat had figured out, pre 9/11, that terrorists flying a commercial airliner in to a building was a risk worth trying to prevent and had passed legislation for all cockpits to be bulletproof cells that could only be opened from the inside, or something like that. 9/11 would have never happened, and that bureaucrat would have caught shit for the rest of his life for championing legislation that is causing a lot of people a lot of cost and frustration and has never had any observably positive impact anywhere in the observable universe.
I will say though, that this is a difficult skill, and it takes practice, because there are so many personal biases that can creep in and lead to a corrupted causal analysis. It's also a bit thankless because once you come up with a decent root-cause analysis, it reads so clearly that it basically comes across as obvious and common-sense to anyone that reads it, leading them to underestimate just how much work it took to get there.
In all these situations, it seems like some of the time, and for a critical part of what you do a lot of the time, you have to have strategies that take into account other people having hidden motivations, and particularly people who don't think strategically and cannot be reasoned with on the basis of strategy, or they refuse to be open with these sorts of things with you in particular, or will be evasive, or say they will do things and then not do them, etc.. A lot of the time you cannot fix these things, you can only take them into account and/or find workarounds. And it seems reasonable to say that this is the part of the picture where you have the most leverage to make a difference, or to reduce things like the amount of bullshit you deal with, or the amount of wasted effort you put into things.
You can also have the insight that other people will see you the same way - as someone who is obstructive, or fails to be clear, or isn't persuing the right goals, etc..
I guess you just have to ask why people have these motivations and why they hide them.
It feels like people would act dishonestly first to seek survival (job security), but then also to seek power (promotion/influence).
I wonder what the relationship is between survival and power. I guess at some point survival manifests into the desire to hold onto power. And the stakes get larger as the power accrued increases.
I also wonder how the human reward schedule plays into it. People are motivated by consistent rewards but at random intervals. If you take out the corporate power politics, you remove the randomness of the intervals, and it perhaps becomes less motivating. The chance for an early promotion provides motivation to play the game harder.
> It feels like people would act dishonestly first to seek survival (job security), but then also to seek power (promotion/influence).
The first place I go to is to see that people aren't always doing good strategical thinking, then they get hung up on mistaken goals, or ineffective ways of achieving them, and aren't very good at communicating what they are doing or hoping to achieve, and this often leads to what looks like subconscious (or sometimes very conscious) obfuscation.
In the technology area, I've worked with many coders, through to product people, sales, marketing, directors, etc., who get hung up on particular technical ideas - and claim these are strategic product or architecural decisions, and the same happens with people getting hung up on ineffective project management - especially when it's an approach that worked in another situation much better than it's working in the current situation.
So it's not about being good at strategic thinking. It's about motivation, boredom and self-discipline.
Here is what you are not seeing:
Dev sits down to work on an assigned task. They might be on the ADD spectrum, and the work feels incredibly boring and their mind cannot focus on it. They are interested in novelty though. This provides a rush of motivation and hyper-focus. It will usually involve the choice of some cutting-edge new technology or architecture. For someone without ADD, it's hard to understand the intense pull of this pursuit of novelty. Tugging at you constantly.
The problem is this motivation is derived from something that probably doesn't align with strategic goals of company. They will have super-human 10x levels of motivation. So they try to push the company in the direction of this novelty. They are just following a high for weeks on end. Eventually this high will run out though and the need to show progress towards business goals arrives, and they can't hide behind "rewrites", "tech debt", "unanticipated problems with new architecture", etc.
So then the survival kicks in and the blame game starts with management.
I think some of this is true for almost every developer.
So when I read your complaints they are all easily explained:
> hung up on mistaken goals...ineffective ways of achieving them
They want to keep working on the novel thing that interests them. They try to shape the goals that allow them to do this. This can be subconscious.
> aren't very good at communicating
Because they have become too side-tracked by this novel thing or the next novel thing.
> looks like subconscious obfuscation
Yep. Exactly. It is often subconscious. To confront the reality of how far you have strayed from the strategic goals of the company would be like being out for a night drinking and at the height of the party, you have to leave and go immediately to bed.
> Hung up on particular technical ideas - and claim these are strategic product or architecural decisions
Just chasing dopamine.
> people getting hung up on ineffective project management
Survival. You went too far off track and need someone to blame.
---
I think if you view developers through the prism of chasing dopamine highs from the pursuit of novelty, you can much better manage them.
The only thing that can tackle this I think is having hard, concrete, and unavoidable deadlines. The survival system overpowers this creative exploration. But this usually only kicks in a couple of days before a deadline.
Your strategic goals are essentially always competing with emotions of a developer towards some other thing. I think a lot of people underestimate the mania feeling that comes from the other thing. So you need to make the the dev believe the current strategic goals and the achievement of them is really, really, really important. This is why visionary/bombastic founders and so important.
What could also help is allowing everyone to be open and honest about their true motivation and goals. But it would be hard for anyone to say: "I am only interested in playing with this new tech, and have no interest in the company". So there will always be a level of concealment. And usually, revealing such things will mean they are monitored more closely which they don't want so it doesn't make sense to reveal these things...or they will feel like they are being monitored which increases stress.
In summary, deadlines and no bullshit.
> I think if you view developers through the prism of chasing dopamine highs from the pursuit of novelty, you can much better manage them.
I think the crux of it for me is that you cannot really ever know someone else, nor can you try control them like a puppet and it work. So you have to accept that you will often fail at trying to manage people in situations like this (and to understand your bosses, etc.), and you need strategies that work in the face of these factors.
But from another perspective, I'm not all that convinced about viewing developers the way you say, trying to indulge them then find a way to use the output. My alternative is to spend more time giving people good context - a programmer may be less inclined to chase their own technical novelty compulsions if you can give them an understandable and non-fake view of e.g. specific sales processes that need to be supported by development and what the significance is, company strategy, what the operations team is dealing with, etc. One of the simple older ideas in this area is having developers sit in on sales/customer meetings.
So I guess this is a kind of argument that alienation is part of the story of why technical people can often be totally unaligned with their company's goals and this is a barrier to improving this.
This may be a perspective coloured by my experience mostly working in small companies, not sure how well it translates to big ones.
Applied to s/w this would be "do we really need to make this system, in the first place: How about if we decide not to"
Saying it was an inattentive driver misses the point that we can't expect fallible people to not make mistakes. We should be designing systems to minimize the damage from inattentive drivers and drunk pedestrians
People rei-ify things. "oh we need to do X we've always done X" when in fact, you could say "doing X is stupid"
Don't passively accept it, or become fatalistic about dysfunctions. If there are achievable changes to make, try to make them within your sphere of influence. Do what you can to produce positive changes, but don't allow frustration to lead to cynicism. Think of layer eight problems as engineering constraints like any other.
Fortunately, we don’t have to characterize this equation to quantify bloat in the corporate structure. We just have to wait for layoffs in the next recession. Unfortunately, this is a lagging indicator.
Many problems don't get solved well (or at all) because nobody in the organization finds them interesting.
The only way to fix this that I've seen is to change the composition of the team and/or change the composition of the problems. This is why so many startups struggle, they have a small team and eventually encounter problems nobody is energized by solving and they can't hire someone who is interested. Eventually people burn out doing things they hate and it all falls apart.
This isn't "five whys" except in the most superficial sense. It can be useful to do this to build reliable technology but it isn't what five whys is trying to solve.
> because the battery is dead... because the alternator wasn't charging it... because the alternator belt broke... because the belt was beyond its useful life but wasn't replaced...
All of these are a single "why" (call it "technical faults" or something).
> It turns out that when you start doing this root-cause analysis and really keep after it, the "squishy human realm" is actually the no-longer-hypothetical "layer eight" from those T-shirts.
Is this actually a revelation to anyone? These hard questions are exactly what five whys is designed to address.
Where are the project/org management AI solutions?
How many chat conversations between managers and employees do you think are actually unique.
I use it for nearly all of the bolts and lags that I need to loosen or tighten.
The bolts on the spindles for my tractor deck are massive, they rust up quickly, and my $200 socket wrench doesn't touch them. For those spindles, I have a special tool that can solve the kind of problems that a $200 socket wrench can't solve.
Despite my socket wrench not solving all problems, I'm still really glad I have it, because it is helpful most of the time, and I trust myself to be smart enough to know when I need a different tool.
If only, amirite?!