Work is work, in which returns diminish (2020)
codahale.com
codahale.com
Reasoning about humans as cogs is inevitably going to be a really awful way to get people and companies to be productive - most of it isn't that. However, humans, like plants, are able to survive and sometimes thrive in really terrible conditions _despite_ those conditions. This is what you see at Amazon; if you throw enough money at anything then you're going to get some kind of result. But, despite Amazon, Microsoft, and Google having billions and billions of dollars to throw at this stuff we mostly still grab for Open Source tools (and so do they). That's largely because the workflows the Open Source community use actually work and that's mostly because they get the human stuff right.
The whole "man hours - overhead = man joules" thing doesn't math and using that as a basis for any kind of analysis is going to lead to weird places. There are elements of human workflows that fit into these boxes - the really mundane work. Reasoning about mundane processes through this lens is maybe a good idea to the extent that mundane work can't be swapped out for more interesting work. The problem is that what we really want is to excel at the stuff that isn't mundane (the force multiplier shit) and that needs an entirely different framework that reasons about humans as humans rather than machines.
One possible tl;dr is:
Objective analysis and queuing theory leads to _rejecting of_ all kinds of Talylorian cogs-and-factory-lines style organizational models and hypotheses in software work.
Here is btw. a book with empirical results about software work perhaps pointing a bit in the same direction: https://www.amazon.com/Accelerate-Software-Performing-Techno...
Intuitively, in smaller organizations these bottlenecks are fewer, and in an organization of 1, your individual capacity is only limited by your physical and mental constraints.
So yes, it's important to keep in mind that everyone's different but there is definitely excellent insight into how people work together effectively in large groups.
A companys maximal performance is limited by this internal eco-system and its external eco-system. There can be short term gains as the system spins itself appart, but there is a peak antelop even for the most well adapted lion.
> What happens inside the boundaries is important
> That we know some of the boundaries of organizational performance and their dynamics doesn’t excuse us from using our empathy to build humane organizations. Companies are groups of people being compensated for having to spend some of their finite lifetimes not being with their partners, children, pets, or super weird hobbies. They deserve to be members of organizations which honor that time by ensuring that their work has value and meaning. There is no mathematical model to guide us to that goal.
The older I get the more I admire people who are straightforward and cut through the gloss. If something is worth saying, it's worth saying simply.
We can have a discussion about whether it’s good delivery of that style, or whether the style itself is accessible, but I’m not sure what we gain by trying to analyze someone’s personality through it.
For example, his phrase cum hoc, ergo propter hoc is a pun on post hoc, ergo propter hoc, which is a well-known logical fallacy.
English being a Germanic language is why the whole "no prepositions at the end of sentences" thing never made sense. That was a Latin rule. But Germanic languages have a property of augmenting verbs using prepositions. In German itself these are called "separable verbs".
Ich stehe auf. I stand up.
"Up" is not a preposition in this sentence. It's an integral part of the verb. That is, the verb is, in its entirety, "aufstehen" or "to stand up".
Meanwhile, this same phrase in Spanish uses reflexivity:
Levanto me. I lift myself.
ceteris paribus, technical terms are obscure to people outside the field. Using them well is a nice way to bring people into the field, and to signal that you mean the term in a very specific context.
And that paragraph is one of the worst uses of "ceteris paribus" I've seen. For one, I'm not even talking about economics. And only tangentially about holding some variables constant while changing others.
Coda's technical chops are undisputed AND is a well known technical communicator, dont cramp his style, there are few enough of him as it is
Writing for the reader is important if you want your work to be read.
Write to think all you want but don't expect other people to read your unedited drafts.
If so, then you usually don't want to mix it with profanity.
I mean, you can, but it's a style that went out of fashion years ago.
How far do you take efficiency as a measure of communication quality?
Did Kevin have it right when he asks "why waste time say lot word when few word do trick?"[0]
"I leveraged a hammer on these nails" sounds hilariously stupid, but it's quite literally using the word leverage for an action that has leverage, unlike tech blogs.
We actually had a course at our Uni to stop us writing in this style. The Professors had become sick of reading white papers that said something straight forward in this strange Academic language.
I'd be willing to bet this person is very well educated or has maybe has worked in Academia.
Grabbing a "couple of beers" could really mean a "bunch of beers". But a "dyad of beers" is very explicitly exactly two beers.
I mean, it doesn't necessitate you to ken an aleatorial Hellenic lexis.
Yay language!
My sentiments exactly.
You want to avoid single points of failure, optimize bottlenecks, build in redundancy in similar ways etc. Etc. It's a great insight.
Put this way, be careful how much redundancy you add; as it is likely to increase complexity and reduce parallelism.
As an aside this is one of the major goals of Agile. Having smaller tasks increases the potential for parallelism. As well as the more obvious ability to change direction.
Lastly, regarding applicability of distributed systems thinking to people, see https://en.wikipedia.org/wiki/Neil_J._Gunther#Universal_Law_...
which is applicable to both imho.
I think it's possible to have good parallelism within Agile but I don't think it's the Agile that makes it happen.
WTF? This is a bad way to think about the 'division of labour'.
One man cannot ever get over a 10 foot wall.
Two men together can absolutely do it.
'Work' is not the measurement, 'productivity' is.
Even the fleeting thinker, who works only 1 day a week, but shows up every day, may be absolutely critical to making things work. Paid for that input, when it matters, not 'daily wages'.
If the value of new people diminishes logarithmicly, or much more likely, at the square root of the number of people, you must tackle tasks that a smaller team could not. Otherwise you'll be out-competed by a handful of smaller competitors.
Yes, you can do things a smaller team couldn't do, but you don't really have a choice, if you want to survive.
Could you explain your reasoning? This doesn't seem right to me. If you have linearity, this means that every new individual is exactly as productive as all other individuals
If the team didn’t have a facilitator, for instance, some team members may not be coordinating, causing issues with finishing tasks.
Because despite having a factorial increase in communication paths, you still got twice as much done? Are you sure?
You have to ask these questions because at worst something bad is happening. At best someone new brought in new techniques that work better than your old ones. You need the answer in either case.
This is completely the wrong way to think about division of labour.
We started to figure this out 300 years ago for god's sake.
It's not helpful think about our tasks as a series of bits of work to be accomplished that we hire spigot-workers to do. White collar work is mostly not labour in that sense.
Thinking about your organization like a CPU doing calculations is utterly the wrong analogy.
More like band, playing a song - most tasks have some level of optimal participation you don't want to be below or over, and/or it might be incrementally harder to improve quality depending on variables. And 'talent' usually matters more than anything.
If you want to build a cart you need a woodworker and a couple of blacksmiths feeding your cartwright the proper materials. You’re right that five blacksmiths won’t get the cart built faster, but there’s a whole world of items, including in software, that are composites of anywhere from a handful to a hundred specialties.
I don’t feel like you’ve countered my theory about the need for higher order creation. Someone somewhere had to look at how many musical instruments there were in the world and think, “composer” and “conductor”, which not only gave you a job but suddenly put a bunch of musicians to work.
Sure. But the counter example (which is what I believe the article is arguing) is that one woman can make a baby in 9 months, but 9 women cannot together make a baby in one month.
Whether an organisation's "work capacity" is bottlenecked by things like gestation periods which cannot be circumvented, or things like 10 foot walls which can be cooperatively circumvented - is a great question though.
You cannot escape Little's Law for instance. You can change the variance but in the long run the average is what matters.
But I think there's a cap to that as well, if you bring 10 people do they get over the 10 foot wall faster than 2 people would?
No they probably don't.
So from 1 to 2 you've got great added productivity, but at 10 you're majorly wasting resources.
That's where the other advice would kick in, getting over that wall won't get any faster, you can't scale beyond a point, unless you use other strategies:
1. Build better tools/frameworks, like say have a team build a ladder, now 1 person can get over the wall as fast as 2 people could prior.
2. Diversify your offering, have the other 8 work on other stuff that don't depend on getting over this wall.
I think that's all the ones mentioned on the article, but it be interesting to see if there could be more?
Now, maybe it takes 10 people to think about the problem of how to get over the 10 foot wall repeatedly and efficiently, at lower cost, for example to invent a ladder assuming ladder's didn't exist. I'll give you that, this is an interesting take.
But did it really take 10 people, or it's more that only 1 out of 10 people will come up with this great solution? And if so, you could find yourself adding 100 people with no one thinking of the solution and a competitor with only a single person might think of building a ladder. So what's the takeaway for scaling here?
What you're saying is like looking at Amdahl's law (mentioned in the essay) and saying "WTF? This is a bad way to think about parallelism, because what if I could just come up with a better algorithm?" Amdahl's law is not about choosing the right algorithm, it's about how much time you can save by parallelizing the algorithm you have. Clearly choosing the right algorithm is important, but it is a different kind of question.
To see this, consider that adding a sales person to a group of engineers, or vice versa, could have a nonlinear improvement in revenue.
It’s an awful buzzword to be sure, but superlinear gains from functional diversity is called synergy and it’s possible.
Are you referring to the division of labor (specialization)?
This is a fair assumption when we're talking about organizing work within homogeneous teams that have (and ultimately are defined by) a single shared work stream. Every individual person on the team may have things they're better at, more historical knowledge on, or a warm cache, but it's generally assumed that any team member should be able to pick up any task.
A good manger/scrum master/whatever will end up organizing work to take those things into account but it's an optimization not something fundamental.
"... can we determine the supervenience of some set of factors on organizational performance, not just in a particular context but across all possible organizations? That is, are there necessary, a priori truths of organizational performance? [..] As it happens, there are."
I think they, and we, know that 'all possible organizations' does not mean what it says, and that's my objection. It's not a big deal, just inexact writing, or possibly inexact thinking if they mean it, which I doubt. They mean 'organizations sufficiently large to be organized into units with fungible roles'.
"The ceaseless pursuit of force multipliers is the only possible route to superlinear productivity improvements as an organization grows."
If we interpret this as saying that 'as we add another person, they should do something different to the previous person', then my complaint is addressed.
The rest of the article assumes you don't do that.
which discusses sublinear and superlinear scaling of complex networks.
The first law, “sublinear scaling,” is for systems that deliver resources. It means a city with a large population needs only ~80% as many roads, power lines, and gas stations per person as a city half its size. The second, “SUPERLINEAR scaling,” applies to outputs of socioeconomic activity. It means a large city produces ~120% more wealth, patents, crime, pollution, and disease per person than a city half its size.
“Remarkably, these two growth rates, 0.8 and 1.2., are showing up over and over again in literally dozens of city-related contexts and applications,” wrote Complexity Science Hub Vienna in a press release. “However, so far it is not really understood where these numbers come from.”
https://www.santafe.edu/news-center/news/researchers-reveal-...
> Successful new products can be incrementally integrated with the existing products where it makes sense, and tooling, libraries, and frameworks can be developed by force multiplier teams to reduce both time-to-market of new products and the carrying costs of existing products
Yeah... This is why communication is often needed in a full mesh.
Look at major companies (ehem, one in particular) and see if you think their products make sense in their integration.
Like did chat app X team talk to chat app Y team? Likely not.
Larger orgs move slower because they're trying to build coherent larger wholes. And that's harder.
It's not hard to launch 10 chat apps a year if it doesn't have to fit into any coherent whole.
Like do you think DigitalOcean just needs to hire disconnected teams, and will end up with something that isn't a frankenmonster crying "kill me"?
Their VM service works fine. Adding another that works fine is not hard.
Building all services needed for a major customer to "go cloud"... you won't get there that way.
(Not saying DO is doing anything wrong. Quite the opposite. I'm saying they can't just hire 100 parallel teams and end up being AWS)
I can pick out so many parts that seem to be an extremely odd take on things, but this one stood out:
> the total time spent communicating will grow quadratically as the work capacity of the organization grows linearly.
The assumption being that everybody needs to speak to everybody else as the organisation grows, which is a spectacularly daft assumption.
It also underlies Dunbar's rule on group sizes and how organizations scale.
The point is exactly as you say, everyone does NOT need to speak to everyone else-- but you then have to structure the communication channels. This is a different challenge in a 15 person org than a 3 person org. Let alone a 150 person org.
If you've ever been on a team which went from 5 to 8 people, you've seen this play out. God help you if the team crosses 15 people and you don't realize this triggers a phase shift.
This is the key insight for me. If you're adding people to a team but you're not putting them in a position to affect what work that is being done you will be stuck with the linear scale as the ceiling
This helps me explain an intuitive belief that I've always held for a long time, you should have a say over what you're doing. If I can't influence what work is being done and how it's being done, I don't feel like I can contribute much.
I was hoping the rest of the article would talk about how to build an organization geared towards empowering employees, but it seems to focus on strategies for dividing up work and suggesting that companies should just become more companies under one umbrella to try and reach the linear scale ceiling, blegh!
The real problem is that brain-to-brain transfer of information is really hard. It is both slow and lossy. And the bigger the organization, the more times information has to transfer from one person to another.
The most glaring problem is that there's no quantity/quality distinction in the piece. The core theory is:
"The work capacity of an organization scales, at most, linearly as new members are added"
This is trivially untrue because larger groups enable new complexity to emerge. 10 people can't build you CERN, even if you give them endless amounts of time. They don't hold enough information. A city isn't merely a village of 150 people multiplied by a thousand, and so forth. Quantity has a quality of its own and new behavior emerges from changes in size itself.
Factoring work into little modules as the article suggests is the same kind of error behind microservice advocacy. The interesting thing isn't the performance of parts of the system or the sum of its parts, but only of an organization as a whole, including dynamics between it's parts, which in any complex system aren't linear, dynamic or even predictable.
Maybe, remote work is a first step? we have been working side by side (at the same place) for a lot of time, and this change is the first one impact one of the most important metric: productivity.
Writing is the same, and hence why I like it so much.
- The guy was a principal engineer at Mailchimp.
- Assumes he solved the problem of organizational complexity.
- Quotes the Stanford Encyclopedia of Philosophy instead of any relevant operational or organizational research.
- Firms tend to ignore outside perspectives and examples. (which is often true)
- Assumes an organization is a distributed computational process. (it's not)
- Kant.
- The *amount* of work (i.e. work hours) a company can dish out grows linearly with headcount.
- Time for parallel tasks amortizes to zero if more resources are added. (no idea what this means...)
- Assumes organizations only write software.
- Assumes people know or care what "superlinear productivity" is.
- Assumes productivity = added value.
- Says that orgs should optimize for individual productivity.
- Queue theory and King's formula and hand-waving about contention
- Management consultants are bad at writing code.
- (I gave up after that)
I think about this a lot in software organizations. Like most takes, I'm probably wrong.
Keeping work parallel is, well, hard, and context dependent. Not worth commenting on. Keeping groups small is something we have plenty of prior art on; as Bezos said, two pizza teams, whatever. I may disagree with his exact number, and he may disagree with how much pizza I can eat in one sitting, but the gist is there.
Keeping Resources Local is the most interesting one.
The biggest productivity sinks I observe in my own line of work, and those around me, is absolutely quantified proximate to: "dependency on this thing we don't control". Here's an example: Everything has to be run in our one single kubernetes cluster. That cluster has some rules about how things are deployed, to avoid tragedies of the commons, like isolated namespaces and process security levels and whathaveyou.
What's interesting, to me, is how this could so easily and correctly be viewed as either a productivity gain or sink. On the one hand: Teams don't have to create and manage their own kubernetes clusters. Big win, that sucks to manage. On the other hand: we do have to integrate and hold a dependency on the Kubernetes Team or the DevOps team for X, Y, Z of this feature development. That's a negative.
I think it turns out being related to that next Principal From Beyond Space And Time: You can develop force multipliers, but oftentimes the development and maintenance of those force multipliers becomes a non-local resource that every team suddenly gains a dependency on.
Its really easy to see the upside of a shared cluster. But let's say I'm on a team that just needs to get a static site hosted on the internet. Well, rules are rules, we use k8s; for my team and my work, this "asset" is actually a net productivity loss. We never would have hosted our own cluster in the first place, maybe we use fly.io or whatever. You can argue that the standardization or the single-pane-of-glass security controls or whatever ultimately make it a net organizational win, but I think that's diving too specific into the example: the point is that these force multipliers can actually become force dividers over time, as their complexity increases to account for a wider variety of complex use-cases, and left behind are the (significant number of) teams still on square 1 or 2.
I don't want to understate how big a problem I think this is in modern software engineering; I think this is a big contributor for why software teams oftentimes trend toward stagnation. The buzzword is Complexity, but more specifically: every force multiplier is an abstraction layer which hides complexity, poorly, then you hire on new engineers or need to train up in a part of the system you've never worked in, and the abstraction disappears.
I'm not saying any of this to suddenly present a solution. I don't know. Some of the things I think about though:
I know this is oftentimes perceived as a soundbyte, but: I really think the best engineers are those with a really strong allergy to complexity. And, unfortunately, in most organizations this isn't rewarded. We reward "woah that's an awesome, complex system you designed that accounts for fifty-six different use-cases, exceeds expectations". We reward "woah, you were absolutely on top of that incident, you identified that fix super quick". We don't reward "woah, that comment you left on that other guy's RFC about how we shouldn't implement this was extremely well-reasoned, exceeds expectations". We don't reward "that thing you built hasn't had an incident in two years".
Every organization is different; because every engineer is different. Some organizations are better than others. But I think the average isn't allergic enough; and I think its not even close.
Second; I think Bezos' model of "everything should be an API" is extremely good, and roughly zero companies take it to the extreme it should be. Teams should communicate with each other the same way they communicate with customers, in every meaningful way. Instead, what we oftentimes end up with is: oh, you've got an API that you want on api.mycompany.com, you'll have to use the API Gateway that the Service Platform team developed, then use this npm library we published to expose a standard API interface, and register your schemas with...
At a deep enough level, this guidance will start breaking down. For example, with apis: You probably do want your api on api.mycompany.com, or at least mycompany.com; so there's at least some coordination with the team that runs the domain name, and some kind of gateway may be necessary. For websites, ultimately there's one bundle to serve, and as much cohesion in design patterns between pages is a Good Thing. But, similar to the allergy to complexity, I think our industry-wide allergy to inter-team dependency isn't strong enough.
This can oftentimes surface in the very tactile details about how things are developed. Let's say your goal is: a single centralized documentation page for the API. Routes are managed by twenty different teams depending on the product. This is a massive coordination problem. First, we could ask: Does it need to be single and centralized? Or could each team, which corresponds well to a product, get their own product.api.mycompany.com domain name, then they run everything? The prevailing wisdom is: nah it needs to be centralized; and there's that lack of an allergy surfacing again. But ok; We The Company say every team needs to expose a GraphQL API or whatever, schemas on some HTTP-accessible path, register with the load balancer, register with our schema repository, and we've basically slid down the hill toward making this coordination problem as difficult as it possibly could be. What if, instead, we did something like: the team which runs this documentation platform runs jobs which regularly scan every repository in our Github org, looking for a landmark file that says back "I'm a service register me"; and they get automatically registered? That would help reduce coordination a ton! Now, what if that landmark file was, say, just a GraphQL schema file? "As long as you put your GraphQL schema file anywhere in your repository, we'll find it and you'll get documentation instantly".
I think one of the core things that every member of engineering leadership needs to do is: keep an inventory of the "typical dependencies" an average team in your company has on other teams. API gateways, kubernetes or AWS infrastructure, logging pipelines, metrics dashboards, alerting, downstream services, libraries; if a team generally has to interface in ANY capacity with something another team owns, write it down. Regularly revisit that list; query your engineers to see if its grown or shrank; then make an active effort to shrink the size of the list, or the size of every dependency on it. In large orgs, there is quite possibly no more impactful way to increase the productivity of your teams than right-sizing this list (usually down).
"agile or scrum or whatever"
"most explanations ... are crap"
"informs how much shit we can expect to fit in a bag"
"the floor is lava"
"chattier than others, ... communication is essential"
"punt on coherence and just ride dirty"
"put their eggs in a single basket"
"blistering number of new products"
"super weird hobbies"
Scanning the essay, most of it is pretty tame. There are a few uncommon words in an essay long enough for statistically some of it to be uncommon. What exactly is the objection?
I could almost imagine the intolerable sniffles of Slavoj Zizek as I tried to read.
On an unrelated note I see that inflation is hitting $5 words particularly hard :)