Worst-Case Thinking
schneier.com
schneier.com
One of the worst traits of a bad business lawyer is to fixate on remote risks when advising a client and driving a transaction. Negotiations along such lines rapidly enter into a surreal mode that leaves entrepreneurs scratching their heads and wondering, "why can't we just cut through this garbage and get to the heart of the deal?"
This sort of fixation normally adds significant costs to any business deal ("we'll need to thoroughly research all these possibilities"; "we'll need a 3-page indemnity clause instead of the 1-paragraph version"), tends to paralyze deal momentum and cause the parties to shift their focus onto side issues in their deal ("this deal can't possibly move forward until we can absolutely eliminate these risks"), and winds up causing parties to negotiate 50-page contracts where a 10-pager might have done the same thing without necessarily addressing every remote contingency that might occur in hypothetical scenarios where things can go wrong.
A good business lawyer should be neither a deal killer nor a CYA type but should be sharp in identifying risks and have good judgment in weighing them against benefits so as to strike a practical balance that moves deals forward while keeping them honest, with the client making the final decisions based on such guidance. Of course, some deals are incredibly complex, or have very large amounts at stake, and this does warrant more fastidiousness about risk issues. The key is to know which is which and to handle each accordingly.
In one informal discussion session, the instructor introduced his concept of "libertarian socialism", intended to design a political system that maximized individual liberty with constraints designed to mitigate the iniquities that might arise in a system of absolute, inviolate property rights.
The example that he used to justify his constraints on property rights was this: What if X owned and lived happily on his private plot of land, but his enemy, Y, decided to buy up all of the land surrounding X in a 360-degree circle, and then build an impenetrable stone wall, completely sealing X within? X would not be able to leave his own land without violating Mr. Y's property rights, and Y could not be stopped from building a wall on his own land. Y would have effectively imprisoned X, and possibly indirectly cause X's death. Since this is clearly an undesirable result, we are justified in adopting a system of law that allows Y's wall-construction project to be preempted.
Most of the libertarian-minded students began attacking this model by construing Y's wall-building as a violation of X's natural rights, and started constructing a list of explicit abstract rights that could be asserted by X to prevent Y from sealing him behind a stone wall a la Edgar Allen Poe.
But in retrospect, it looks like everyone, the instructor and the students alike, was making the same mistake: in attempting to universalize their model, they take on the challenge of having to accommodate every imaginable contingency.
Perhaps it's not merely the tendency to focus on the worst case at the expense of the general case that causes these problems, but also the tendency we have to think things through in an abstract, categorical way, and rationalize universal solutions instead of defining the particular scope of our solutions from the outset.
"My nightmare scenario is that I don't know my nightmare scenario. We can obsess about all the extreme things that could happen; but the truth is when, or if, it does happen the chance of it being one of them is tiny. In fact we probably haven't thought of the very worst sort of scenarios; those tiny little things that someone suddenly figures out they can exploit and turn against us. We cannot predict or offset these scenarios. All we can, all we should, do is make security relevant and robust - and then be ready to meet threats quickly when they occur."
(pretty much what Schneier says too; but I thought worth sharing)
So we would be better on focusing on the core, rather than the edges.
Yes. I build small apps for various tasks at my current job and am always surprised at how bullet proof they want the thing to be. I spend the majority of my time trying to safely handle the remotest of issues for these apps which are mostly fairly trivial (not operationally critical) if they stop working. I find it especially amusing because we host our entire operation off a single box in our own network closet.
In other words, make sure you take care of all the simple things that have a much higher chance for disruptive error before you start worrying about the once in a lifetime catastrophic ones.
A better approach to risk management: create a table with likelihood on the vertical axis, and severity of outcome on the horizontal. Then look at the costs to move any given risk either down or to the left. Do some cost-benefit analysis on reducing your overall risk profile, and focus your spending accordingly.
I think a better approach to risk management is to flip it and think of the key parts of the system, and figure out ways to make the system more robust, have redundancy, and some strategic slack built in.
But the real solution is to distinguish risk avoidance against risk response. Prescriptive solutions tend to focus on avoiding risks, and building those avoidance methods into the baseline system that applies generally. But if we isolate the unlikely risks in the way that lotharbot suggests, we can draw a line across the likelihood axis and say "we'll avoid the risks on this side of line and deal with the outliers on a case-by-case basis."
The best strategic slack is to have a robust way of handling unexpected occurrences after the fact, so you optimize the prescriptive system to the expected general case.
BTW, I'd bet there's a pretty strong correlation between "risks that we can't account for" and "risks that are unlikely to occur".
"What happens if you get hit by a bus?"
I have no good answer, but I have lots of bad answers:
"Can't happen. There are no buses where I live."
"I work out. The bus will get damaged."
"If I don't log in 3 days in a row, the server will automatically ftp all the source code to you."
"No problem. I have Blue Cross."
"No problem. All the code is self-documented."
"I guess I won't care anymore."
Now imagine the situation in which that guy is trying -- and failing -- to make his bosses care about that...
It's still correct to spread knowledge around as much as possible. First, it prevents a single person from becoming a bottleneck in development. Second, a better understanding of the rest of the system helps developers understand their assignment better.
Put it this way: by hiring an unneeded developer you are betting circa $100k a year that you will be hit by a bus. From that point of view it seems like a losing bet.
It's not just "hit by a bus", "wins the lottery", or "decides to quit". It could be: "takes a vacation". "Gets snowed in" (generic weather disaster that cuts travel and communication). It could even be "sick".
The bet looks a lot worse when you realize it's just saying "this one guy won't so much as catch the flu".
And hiring isn't the only corrective action. Making time for documentation, automation, cross-training -- these are all significantly cheaper and still tip the bet well back into your favor.
Inside an existing company I would totally agree with you. I'm not in any way advocating for the job-security hoarder of information.
Being a freelancer, when I've been presented with this question it always seems like they really mean "but we could hire 5 coders from massive outsourcing company X for the same price". It is frustrating to try and convey the idea that, even when various metaphorical bus accidents are taken into account, the large well known company is not necessarily the safest option.
No amount of documentation or automation will assure someone who really just wants to be able to say "I chose IBM, it's not my fault it went wrong".
"Well, what would happen if you were hit by a bus?" or "what would happen if an asteroid collided with our building?"
I.e. illustrate the silliness of designing your solution around statistical outliers.
http://wtop.com/?nid=25&sid=1956201
Even if the people paid to think about these problems mean well, sometimes the cure is worse than the disease.