If we had the best product engineering organization, what would it look like?
jamesshore.com
jamesshore.com
There is a lot to not like about Meta and Zuckerberg, but saying he's a bad business man is a little silly. Metaverse was a wrong and expensive move, but it was a wrong move they could afford.
It's perfectly fine to take a bet that you're not 100% convinced will pay off (professional poker players and traders understand this on a very deep level), as long as the potential upside is massively larger than the downside.
Zuck understood that Meta could take the hit if the metaverse bet didn't pay off, but that they'd be massively better off if it did, and they had their own platform. Apple's blatantly anticompetitive behavior around ATT was a prime example of what happens when your business is reliant on "platform overlords."
I'm not fully convinced that the metaverse era is over, though. If they can get Orion costs down and put something of that quality into serial production, I think they still have a chance there.
It was FOMO. They had no vision, they had no plan, it was clear that it was only a thing because it was a buzzword at the time. Just like every other company stuffing crypto-adjacent things everywhere. It might have been a bet, but it was obvious that it was a really terrible one.
> Just like every other company
What other companies (besides Apple and a few startups that were specifically in the metaverse space) went in on the metaverse?
I think this was very much unlike AI and crypto, where everybody wanted a piece of the pie. Meta seemed a lot more invested in this than most of the other tech players, which makes FOMO an unlikely explanation to me.
It's okay to dream. It's not okay to burn billions of dollars on dreams with no proof of concept or business plan. Set aside the question of whether or not it's a bad idea - that's just plain bad execution.
Unless the CEO is psychic, they’re necessarily going to make a lot of bad or wrong decisions. The key is being able to recover quickly and move on to the next thing. A bad CEO makes no big decisions for fear of being wrong.
When FB bought Instagram for $1B, there were a lot of talk show hosts riffing on Zuckerberg for making one of the stupidest business decisions of all time. A lot of executives who got to their position by corporate ladder climbing have personalities that would be terrified of that sort of widespread criticism. They would never make the kinds of decisions that might possibly put them in the unenviable position of being made fun of on national television.
Take Meta's acquisitions of Instagram and WhatsApp under Zuckerberg. While these proved strategically valuable, framing them as evidence of unique CEO insight misses a crucial point: Many others in the organization likely would have made similar choices given the same position and information. The success stems more from the concentrated decision-making power than from individual brilliance. What's fascinating is how we conflate organizational structure with individual capability. When good outcomes emerge from hierarchical systems, we rush to credit the person at the top rather than examining how the structure itself shapes and amplifies their decisions. This creates a self-reinforcing cycle: hierarchical success is used to justify more hierarchy.
But imagine if we distributed decision-making power more broadly, tapping into the collective intelligence and diverse perspectives of entire organizations. Research on collective intelligence and successful worker cooperatives suggests groups often make better decisions than individuals, especially on complex issues. Companies like Valve and Morning Star have demonstrated that flat organizations can be both innovative and profitable. The real opportunity lies in reimagining organizational structures that harness our full human potential - not just that of a select few at the top. By questioning our assumptions about hierarchy, we open ourselves to discovering more dynamic, equitable, and effective ways of working together.
If #2 is true shouldn't an organization like Valve run circles around every single competitor it has since they're all dinosaurs with hierarchies and Valve isn't?
Also, sometimes people say things like "Only [ceo's name] could run [company]. Look at their results!" This is just survivor bias. Who could know whether someone else could or couldn't run Facebook (or Tesla, or whatever)? Who can say with certainty that out of the 8+ billion people on the planet, only one particular guy could run the company, and that particular guy happened to be the guy who indeed ran it? What an improbable coincidence!
Buying IG was a good move because it has paid back Facebook's shareholders multiple orders of magnitude. Isn't the goal of the CEO to steward the company and to make shareholders returns (wether public or private)?
The CEO's job is to decide on the future direction of the company, and then convince both the owners and the employees that this is a good direction. The first part is easy to replace with AI, the second part isn't. At least not for now.
I guess taken to the limit, the CEO will become replacable the moment that the employees have been replaced with AIs, because that that point there's nobody left to lead really. One could actually argue that this is tantamount to the CEO cutting off the branch they are sitting on. After all, once all the employees are AIs, what's to stop the shareholders from saying: "Hold it right there Steve/Jeff/Mark/etc, why are we paying you big bucks? You can be replaced with an AI that will make much better decisions, and there are no employees left to lead anyway."
This seems very software-centric. You can do this today - e.g. Red Bull famously outsources basically everything but marketing[0], so they already have very few employees. However they do have a lot of suppliers, and that all needs managing.
[0] Their marketing is either simple TV ads or incredibly complex stuntwork and extreme sports.
Amazon does a good job of training new hire on the 'Amazon way'. Amazon does 6 pagers, they do design docs. Amazon does SOA. Amazon does not use relational databases. Everything has an API. Because of the 'Amazon way' and the training they do new team members understand at least some of the context and expectations.
Is it the best way? Probably not but no one knows what the best way is anyway. At least they have a way. Saves a lot of effort compared to every new hire relitigating the process and architecture.
Huh?
Somewhere along the same timeline, the operational recommendation for teams was to not use relational databases.
https://www.theregister.com/2019/10/16/amazon_ditches_oracle...
Of course there are relational databases running OLTP workloads, but it’s far away from the norm. There was a program a while ago to shift many RDBMS systems onto something else.
The theory is that with rdbms you have a magical box that scales vertically until it doesn't. And when it doesn't all you can do is scale back the customers until you fix it with sharding or a re-architecture. Basically you tend to hang yourself with indexes and transactions. Also generally when an RDBMS fails it fails down to like 30% throughput.
I usually have an explicit DENY for dynamodb:Scan for the IAM role used to access the DDB table
I’m not sure about AWS.
As a counterpoint, a huge part of Amazon's culture (or at least, AWS's) in my experience was the emphasis on operations and the fact that they didn't have any separation between SREs/on-call engineers from the people who implement their services, and at least for me as someone who had never been on-call before in any meaningful capacity (due to my previous job working on a libraries rather than services), the training for it was basically non-existent on the two teams I spent time on. The "training" I did receive essentially consisted of being put on the rotation once to shadow, where I was able to sort of see what the actual on-call person did but didn't really have any explanation for how to know how to do them other than being told to read the runbooks, which were not really written in a way that was easy to understand for me as someone who was so new to learning all of the internal AWS tooling and ops in general. The next time I came up on the rotation, I was expected to be able to manage on my own, which essentially meant that literally no matter what occurred, I ended up having to escalate because I wasn't knowledgeable enough to fix literally anything within a timeframe that would have been reasonable.
Most people seem to think it's fine for companies to offload the entirety of the burden of learning to individual employees, and maybe I'm an outlier in this regard, but to me, this seems more like a cop out to avoid trying to actually solve the problem at the cost of the employee's emotional health. I'm not surprised that companies default to this, but it's also not surprising that burnout is so common in our industry when this is considered the "best" or "only" way to do things.
I.e., treat DevOps as a way of working, rather than a role meaning something akin to "Ops person who knows terraform, or k8s, or Ansible etc".
This is false, at least in my very thin exposure to the company: I interviewed for a team last year which was maintaining EC2 SSH keys using MySQL.
1. Wherein do you find the smugness? It does not speak to any person, let alone the first person.
2. What would give the impression that comments on the internet are written for others? Carly Simon once recorded a popular song about this type of falsehood.
3. It remains that SQL isn't relational. That is why it "won", after all. Relations are too complex for the layman to understand. Tables are much more familiar to the people in charge and arguably a better model for most business problems.
Just the section on how to rewrite alone communicates something incredibly valuable that grumpy-engineers like myself have great trouble getting others to understand.
I don't have my mind made-up on XP, I've never worked at a place that actually supported collaboration (often workers spoke different first languages, vastly different experience levels, had minimal social graces, were uncomfortable asking questions), but I think it could exist with great effort and would have a lot of upsides.
I've worked at various places where this was supposedly the system.
Guess who had budgets, hiring powers, went to leadership offsites? Yeah not the ICs. It usually just means the C level will smile and nod while "listening" to your feedback instead of ignoring you completely
Has anyone worked at a true inverted company where centuries of classical power structures are thrown out the window?
I feel it can never be properly implemented unless in eg a cooperative
https://arstechnica.com/gaming/2020/07/valve-secrets-spill-o...
To be clear, those are strategic decisions "what are our goals and how do we allocate resources to achieve them". Tactical decisions are "what specific actions do we take to use available resources to achieve goals".
"We’re an inverted organization. That means that tactical decisions are made by the people who are doing the work, not managers. (In theory, anyway, we’re not perfect.) So we’re looking for people who have peer leadership skills, who are great at teamwork, who will take ownership and make decisions on their own."
They usually repackage people’s work around them into their own, take ownership and defend loudly their territory (project ownership) and methodically build relationships with leadership. Having “leadership skills” and being good a team work are often orthogonal to each other.
No, it's actually people who can put together a technical plan and drive its execution with other engineers, or who can clarify complex problems esp. when there are conflicting opinions, or who can see problems before they become disasters and organize the right group of people to take care of it...
There are many examples of leadership, which have nothing to do with the sour view of managing-up or taking credit for others' work.
If you're not lucky enough to have management that's exactly technically aligned to your project someone has to be managing up and paying attention to leadership or else expectations will be totally off from reality.
[1] https://devblogs.microsoft.com/oldnewthing/20091201-00/?p=15...
"FAANG" isn't a "level", it's just a cluster.
* MVP doesn't count.
** Can include users inside the organization.
*** It's OK if it requires senior-level ongoing support. I think expecting it to be maintained by monkeys is a bad idea.
I also worked for hardware companies, where shipping stuff had some pretty serious stakes, and learned how to make sure we got it as good as possible, before getting it out the door.
I like the idea of evolutionary design, and "tuning," but I think it's a bad idea (for me) to deliberately ship bad software as an end-product.
(Also, MVP, by definition, generates lots of trouble tickets. I am allergic to trouble tickets. It's totally a personal thing, but I live by it).
In fact, my way has been working for me, for decades.
I'm quite aware that many folks do it differently, and that's one reason that I try to "keep it in the I," and write about how I do it, and talk about the bar that I set, for myself.
Most of the software I write, is free software that Serves a pretty small demographic. It can have a fairly outsize influence on the lives of the people that use my software, and I really care about the end-users of my work, so I tend to set a pretty high personal bar.
I'm quite aware that I don't have many of the stressors that beset commercial software houses, so I sincerely don't feel "snooty." In fact, I feel profoundly grateful to be in a position, where I can follow my muse.
I really would like it if folks wrote better stuff, but I am also aware of the culture, and how that's next to impossible, these days.
Maybe engineer #1 is constantly pushing up code. In the time it takes them to merge 15 PRs, engineer #2 opens only 1 - but maybe they thought really deeply about the problem, and their approach actually saves the team hundreds or thousands of hours of future development work vs how engineer #1 would have solved the problem.
Part of what makes this so hard to measure is the long tail effects of development decisions. (Incidentally, that's also a source of burnout for me - the constant mental overhead of worrying about the long term implications of what I'm doing, and particularly how they effect other people. It's very challenging.)
There are some core areas of the application that are much more important, but they are often the earliest data structures and built before the problem is known. You will not know how your code will change, so make it as consistent as possible with the rest of the system until you know more.
The issue is when metrics are used to stack rank teams with no thinking put into it. You can't treat correlated metrics like direct metrics. A logger might be evaluated based on how many trees he cut down in a day. There is no comparable way to pay software engineers piecemeal.
Metrics are good, but people want to use them without thinking or taking context into consideration.
The most productive fpga engineer I ever hired was so hopeless with git that I had to hire a second software engineer to babysit him.
After I left both of them got fired and the product they were ahead of schedule on when I left had slipped 2 years behind before it finally got cancelled three years later.
"They'd beg to work for us" - what the f8ck.... if they were the best they would not beg anyone how degrading...They would be there for a mission or wanting to improve something about themseves or other parts of the world.
There's nothing here apart from Agile coach wanting to get some more work.
1984 was released in 1949, if anyone thinks these words / values really mean what is writen wow. People, Internal Quality, Lovability, Visibility, Agility, Profitability...
Overall I don't know if, in the context of a staff developer, he's vastly more productive than say, another dev I have who produces less but levels-up his team better than almost anybody I've ever seen.
But it's also depressing to see how good things could be and how poorly (IME) most orgs are run now. I know I've seen the exact 180-degree opposite of almost everything mentioned here: no team leadership or empowered people, no clear path to the next level for those interested, lack of communication, no emphasis on internal quality, overall pathological product choices (or lack thereof) and on and on. I'd kill to be part of an org that puts this much thought into everything.
A fundamental mistaken belief.
Who wants to pay for the very best people when the 97,000th best person will do? Also how can you decide who the best people are when you can’t even measure their productivity?
With the ability to share screens/IDEs remotely the need to be at the same desk may have shifted, but working together is intrinsic to pair programming I believe.
The original text went into some detail about making the desk work for 2 people, and having screwdrivers available to do so, which for some reason always amused me.
This is extended to "mob" programming where you have whole team of "navigators" and one person at keyboard.
I especially liked the simple 'career ladder' example, for a) focussing on mostly on behaviour rather than knowledge, and b) for being simple to use and track progress with. (I've never seen anything like it in any of the large organisations I've worked in to date.)
This is sort of like your girlfriend asking you "how much do you love me". Except if you answer wrong, it's still more likely that your girlfriend will stay with you than that you'll keep your VP of engineering job.
In most large tech companies, VP level people are so detached, delusional, and unskilled in engineering, that they end up undervaluing what engineers really do. They are unable to explain it beyond stack ranking them.
As an example, this post talks about how simplicity and maintenance brings value. But my VP literally fired people who did not produce new complex impact.
Just goes to show why so many people hate the big tech industry as employees. It is being run by charlatans who abscond from any real leadership.
Indeed, they list Quality immediately followed by customer happiness (“loveability”) which is aligned with XP, the practice supported in the article.
The agile manifesto isn’t the only way to deliver good results in software.
Why not just provide a clickable link given this is an article on the web?
Because the images are slides from a presentation that the audience could scan.
>Thank you for listening.
The text of the article appears to be the "talk-over."
https://www.jamesshore.com/v2/blog/2024/update-on-software-e...
What about all the words that are not picked? That's for 2 years later, when they play the same game again.
Here's the last but least favorite part: beat employees into memorizing those words, have them graded against these words based on entirely subjective interpretations, and reward those who are good at playing this game.
Some people are just story tellers, not doers.
Then I look at the profile of the author of this blog. Yeah, makes sense.
How do you measure productivity = how do you decide whom to promote; how do you decide whom to fire; how do you decide how to distribute bonuses; etc.
If you can’t measure productivity, you can’t do your job as an engineering manager. It’s not a question that should have been asked 3 months into a job. It’s a question that should have been asked during the hiring interview.
I'm sick of these (implicitly) absolute measurement questions. I pretty much refuse to look at anything other than the delta.
The qualitative measures you mention describe "quality" rather than "productivity" (rate of good output). Both are aspects of performance, but are definitely distinct.
I suspect that the best you can do for productivity is kind of a halfway house, where - as part of their feedback - a more senior developer indicates whether the rate of implementation met/exceeded/fell below expectations.
Why not? You could feel productivity instead, which is also arguably the better approach as it more emotionally appealing to those paying the bills than measurement is. That is how people do it in the real world. Nobody is measuring productivity in this business, even if they want to pretend to themselves that they are.