The Developer Coefficient: a $300B opportunity for businesses
stripe.com
stripe.com
The report is basically leads one to conclude that with well developed API, the world would be more productive if we can outsource common business logic (like payment processing) to we'll encapsulated and designed services. Why? So you can spend more time building new stuff instead of fixing stuff that many other people are also fixing in their own implementations.
Basically, I see this as a "code is a liability, so use a SASS; we aren't biased in our framing of it"
90% of code is just reinventing the wheel, over and over again. And not just specialised modules like payments. Its insane that even small companies have "data experts", "UI/UX designers". Best practice is actually quite narrow and less important than we think, better to have 30 great designers working on the best way to allow users to interact with devices than 30.000 wasting time. Likewise, its better to have a bunch of smart people working on abstracting away data storage then everyone digging through AWS docs.
The truth is, WE have not been able to agree on standards of software development that would see the complexity abstracted so we can write applications faster. And because of this, cloud platforms and SAAS providers are filling the gap and increasing in power.
Every time I watch the mother of all demos [1] I feel depressed. Its as if builders were given the power to create new construction materials from scratch, for free. And then spent the next 50 years arguing over whose material is better than actually building cool stuff.
I disagree. More than 99% of code doesn't require to (re)invent anything at all. Actually, if we stopped trying to be too clever, it would be much easier to clean and refactor code.
Edit, to be more specific: mechanic, repetitive, naive code is maybe boring to write, but easier to read and understand.
I ask because that fits with my reading of both the report and the rest of your comment, but a few replies seem to interpret it as cynical.
At the very least, the report quantifies to some extent the cost of code.
I'd love to see a more in depth study that deals with integrating with services vs libraries vs written in house - each of those have costs, though probably not equally.
As software continues to eat the world, I am reminded of Uncle Bob's blog on a Hippocratic Oath for software developers and the gravity behind the consequences of what we produce.
http://blog.cleancoder.com/uncle-bob/2015/11/27/OathDiscussi...
It's only a matter of time before the discussion changes from "Developer Coefficient" to much more dire tone. To quote Bob Martin,
"With that great power ought to come great responsibility. And, indeed, society will hold us responsible when our actions result in disaster. And yet nothing binds us together as a profession. We share no ethics. We share no discipline. We share no standards. We are viewed, by our employers, as laborers. We are tools for others to command and use. We have no profession."
I've personally found it very difficult to discuss ethics and professionalism. Many SWEs either don't care or see it as an opportunity to try to inject their personal ethics. Neither is helpful in formulating a professional ethical system.
Unfortuantely, it is harder to measure how "good" things should be in software engineering compared to the need for a bridge to stay up or a doctor to not make somebody sick.
I could see governments introducing laws that say every company who writes software need certain credentials etc. but as we already know, ISO registrations or PCI certifications do not actually mean the people who do the work know what they are doing well enough.
I consider myself a pretty ok developer, in the sense that most of my time is employed at re-writing systems that have been written poorly by employees. So instead of writing new a shiny software, my valuable time is spent replacing something existing with something else. Zero value? Not quite!
The first-written system has 2 objectives: 1. getting something out to validate the business model quickly - 2. Refine requirements following business needs.
Point 1 implies that for the original system: the sooner the better. Definitely at the expense of quality.
Point 2 implies that the system will experience incremental engineering, often leading to messy code.
But, the market will be validated and the requirements refined. That definitely has a cost, happily paid by technological debt.
That's actually why it's called technological debt!
Yes, often. But not necessarily. The worst problems I've found were caused by bad abstractions. Naive code is seldom the source of troubles.
Otherwise, I would say most of what they're looking at (what is bad code, how productive are developers, etc.) is so hard to measure, that the opinions of people on them are nearly useless. There may be a company or two out there that has a good handle on the productivity of their developers, the level of technical debt in their current codebase, etc. But surely most of them do not, and thus an average of the opinions of many people about something they have no clear handle on how to measure, is worse than useless.
> Which of the following technology trends, if any, are having the greatest impact on your company in 10 years?
> Blockchain: 0% developers, 20% C-Level
little C-level execs answering this positively understand what blockchain is, and if they said blockchain will impact them in the next 10 years, a lot of that comes from the BTC hype. This also is shown by 0% of developers thinking it will have an impact.
And the big ones (e.g., Bitcoin) are more properly trees, though mostly all branches except one will be ignored.
Blockchain: - developers, 20% C-Level
I didn't interpret "-" as "0%". To me it seems more like they didn't get data for this particular trend. The same thing appears in the next line for ML.
ML: 20% developers, - C-Level
Does it say anywhere that "-" means "0%"? Surely some C-Level execs think ML will have a big impact on their company.
This is obviously shortsighted for reasons most developers already understand.
It's an extreme point of view, sure, but in a perfect world this would be the case every time you build something.
On the opposite end of the spectrum you have to rewrite the entire codebase every time you interact with it to make it fit.
Reality is somewhere between those.
Yet the same people who hate devs spending time on refactors also laugh when you give them a six month estimate for feature X. I think "perfect" is "good enough" in this context. "Good enough" is always preferable to "non existent yet theoretically perfect".
I believe that what is missing more than anything in our industry is the veterans who have gained knowledge and wisdom through both the breadth and the depth of their past roles and can now do better than the previous generation. In real engineering fields or other scientific disciplines like medicine, there is a culture of learning from experience, and the people doing the most challenging jobs often have decades of it. In software development, we laughably call someone with 5 years of experience split between 3 different jobs "senior", and a lot of developers in their 30s are already looking for an escape hatch before ageism halts their career development.
So another good way to leverage developers might be to improve their working environment so they don't all quit just as they're starting to figure out what they're doing.
The real issue is there is not 1 correct way to write software but although we are understanding programming more than ever, we are still not great at quantifying and describing the values or requirements that drive our choice of process or even our selection of the right supplier to write it for us.
We had a bad experience from a supplier, not because they were rubbish or lacked skillsets or ethics but because they were not a good fit for the type of product we were building.
Sure. But they all move slowly and deliberately, mostly through careful evolution of good practices as new evidence and reasoned analysis become available. We don't throw out everything we've ever known about how to build reliable bridges every six months because someone thinks that suspending a paper bridge 200 sheets thick from orbiting satellites instead would be cool.
True game-changing developments do happen in software development, but they are quite rare. Most progress in industries like web development is illusory, and it's only a successful illusion because they people who have been around for a while and seen 99% of it before in other contexts have left.
Great, then we can save the money spent on version control as well!
I actually had someone suggest this to me as I was installing a VCS. ("Why don't you just write it correctly the first time?")
Can we please find a way to automate these C-level executives next?
Us mere mortals could never understand them, let alone code them.
18 000 000 Devs worldwide
* 51 000 USD Global Domestic Product per dev
---------------
918 000 000 000 Global Domestic Product of all devs
* .316 Percent efficiency loss per survey
---------------
290 088 000 000 USD loss for all devs globally
I'll wager that we're seeing the result of management practices that don't properly leverage the ability of computers to bring competitive advantage to organizations. Instead, we see organizations struggle with the need to totally rethink their approach given the powerful tool that computing brings. Furthermore, my bet is that most organizations won't successfully transition, and that a new breed will take over.
Wow, there's a $300B untapped market for developers who write perfectly maintainable code on the first try and produce no tech debt?
Sarcasm aside, there is a difference made between bad code and (regular) refactoring and tech debt in the report. I read bad code as the mistakes a junior might make but any reasonable experienced engineer shouldn't anymore.
That being said, most software today is written without the benefit of quality code analysis or regular linting. There's likely a distribution problem to be solved as well.
Yes, just like anything there is inefficiency, waste and even some corruption but that is the case even for building bridges or laying 3 lane roads.
Maintenance is actually the most non-trivial part of any software esp. successful one and to discount it reflects the lack of sophistry on part of the actors who reside in C-suites.
I mean, if you did say that, you'd have a point. It would be smaller than claimed, but there really was an untapped market opportunity.
You can go to Amazon right now and buy cheap ebooks from self-published authors who obviously haven't had any editing done. The quality of writing in these is so poor that I've stopped being willing to buy self-published books, but the books do make sales. Completely eliminating the overhead costs of editing (because you don't do it) and publishing (because that's now cheap enough to be free) has opened up a market for low-price, low-quality stories.
Which is the entire point. Unlike books, software systems must continue to function and grow for years. You read a bad book and you toss it aside. You can sell bad software (which will never be fixed), but there's not really a market for it.
I actually expected a real potential market, silly me.
If $300B is burned to maintain and refactor new code, that's $300B of value to capture with better tools, infrastructure, and theory. Now, if that were actually to happen, there would be an even larger sum of money burned to maintain and refactor new code, because we just all got $300B more efficient and that means more effort is spent applying software to new fields, industries, and markets which will in turn require brand new code to go through the wringer again.
Any reference or data to back up this assertion? My intuition would say otherwise, if for no other reason than that because the many layers of abstraction that exist today allow a person to become productive writing code with significantly less knowledge and in a shorter time than in the past. When you combine this with a general shortage of workers in the industry, you have a huge influx of individuals less educated in the field and with less experience. With all that considered, I find it hard to think that we're any better at writing software, in fact, the abstractions that exist today are all about letting us be worse at writing code while still being productive.
This is far too broad.
We have better tools (compilers, static analysis, linters, etc.) to find some of the errors we make earlier in the lifecycle. These provide the ability to rapidly address categories of error that were once avoided primarily by an arduous and slow process of scrutinizing hand-written programs. Coupled with more abstract programming languages and computing resources, what we have today is the ability for single developers to write incredibly complex software quickly--with bugs that have a complexity that increases non-linearly.
Sadly, never managed to get the company or anyone to publish the fix. Last I checked, the bug was still present, it's survived at least one full redesign/rewriting. The losses are well over a billion dollars by now. :D
Not refactoring is like being a file clerk who saves time by never putting anything back.
> How much of a priority is it for upper management to increase the productivity of its developers?
> High / medium priority 96%
Every single business function suffers from process inefficiency, rework needs, upkeep/maintenance needs, incremental changes/adaptations over time, and quality differences between deliverables/execution from senior vs. junior staff. The only difference between those functions and IT is that:
1. IT (and developers in particular) are incredibly expensive resources. Even junior employees here can cost more than seasoned professionals in other departments. This amplifies the scrutiny it receives.
2. IT work tends to get "projectized" more than other business units. This puts a higher level of visibility into time usage, allowing more granular accounting of where people are spending time.
3. Developers (as opposed to most business users[1]) have more exposure to "the right way"[2] to do things. This allows a level of self-awareness of process debt that leads to dissatisfaction and grumbling whenever they have to do things that wouldn't have been required if the process debt hadn't existed. Which translates into telling management that they could have done more if they had done things "the right way"[2], which management then interprets as lost productivity. But doing things differently would have necessitated a different cost outlay and timeframe to the original project, so in reality it's just shifting costs back to the project itself (or diffusing it into operational support) rather than post-project maintenance costs.
--
Process debt accumulates everywhere. Everyone knows it's harder to get things done in an enterprise company than a small, scrappy startup. That's not just because the technical systems are newer and unencumbered by legacy systems. It's because the entire business is newer and unencumbered by legacy processes, calcified skillsets and policies, or what "normal" is. You can assimilate best practices and newer approaches to things because there's no internal inertia yet on anything, which includes but is not exclusive to the technical components.
You can shift around those $300B in costs, and it's mostly an optics issue. Mitigating technical debt isn't free, and has no silver bullet. The cost is just shifted to the front end instead of the back end, in the form of operational support resources, process changes, automation, and hiring better skilled employees. It could well have ancillary benefits in the form of more execution agility over time, but from a cost perspective that $300B is just going to shift elsewhere, not go away.
Developers do themselves a disservice by derisively speaking about technical debt to managers, and pointing out that if they could have only done things differently, that technical debt wouldn't exist. It paints the perspective that the technical debt is the result of individual developers decisions/actions, rather than a symptom of operating within the existing operational and organizational processes. If your company runs projects with a high level of rework needs and maintenance afterward, that's the result of organizational issues beyond a single developer and should be viewed as an expected outcome that rolls up into the cost of executing projects in the existing environment. It's a separate conversation about whether to invest in process and operational improvements to reduce those types of project costs.
[1] I've set up entire drip marketing programs for companies for ~$5k before, with minimal ongoing costs for maintaining it and passing through email platform costs. I've also seen instances where companies have paid $40-50k to have an agency create a single "thanks for signing up" email, and pay $40-50k to create and send a single quarterly newsletter email to less than 10k recipients. And they didn't consider that a waste, because after accounting for their accumulated internal process debt, a three month lead time and $40-50k in costs was a competitively bid rate to execute the work for them.
[2] I put it in quotes because "the right way" is usually not, in fact, an objectively correct way to do thing. But rather an ideal best practice or a following-the-bandwagon approach, without any critical thought or evaluation on whether it truly makes sense to apply to their environment.
Knowing the rough costs of rework and bug hunting, I would hope that "studies", like the above, are taken to add more weight to the activities in software engineering, not just the programming/coding aspects.
How is this tracked? Looks like it's just a survey of opinions. From my data I see the median time spent in IDE at 11 hours so 17 hours spent on refactoring must be a lot of non-coding tasks.
In my case, I hardly do any software engineering. I spent a lot of time on meetings and doing non-technical work for an army of managers, PMs and other leaders that have 100% of time availability to spend furthering their careers.
Yes, the underlying principle of improved efficiency having benefits may be true - but we knew that without needing this report.
It doesn't seem unreasonable that most people should outsource work to a supplier who can specialise in something but it does not, of course, answer the questions about the skills required to manage the outsourcing in some cases. For things like payments, most people outsource anyway and the domain is relatively narrow so there aren't too many gotchas for most people.
Stripe, first of all, your report misrepresents software development reality.
You present software maintenance as a massive waste of time and money. But that’s just not true. Maintenance is how software stays relevant.
Laws and regulations change, technology platforms change, features rise and fall in importance, the users themselves change over time. There are endless justifications for ongoing software maintenance.
For as long as software serves a useful purpose, it will need maintenance.
Yet you present maintenance as somehow bad, when in reality maintenance is the natural consequence and an essential aspect of software development, and it has always been so.
You’re also pandering to an ignorant attitude that believes software can be written free from bugs and perfectly formed from the outset. Thus all it takes, thinks the ignorant executive, is for my coders to be better than they are.
You highlight the cost of “bad code”.
You don’t provide a definition of bad code, so here’s mine:
Bad code is code that is hard to understand and/or hard to modify without breaking things and introducing errors.
If you know software developers, you know we are, on the whole, vocal and assertive about the avoidance and elimination of bad code. Look at what we talk about at conferences, in blog posts, and on twitter.
Writing and learning about good code is a consuming passion for many developers. Search for “clean code” to see countless examples. How do we avoid writing bad code in the first place? That is a billion-dollar question (or $85bn as you say).
Everyone without exception agrees we should not write bad code, yet bad code persists.
Bad code is a function of ambiguous requirements, unreasonable deadlines, lack of training and support, lack of a proper testing regime, lack of appropriate project sponsorship, internal politics, lack of funding, and so on.
Yes, some engineers will at times be lazy, thoughtless, short-sighted. Just like their managers. Just like their manager's manager.
But in the round, bad code exists primarily because of human, social, and political problems we all share.
And then consider how we eliminate bad code. We do that with maintenance of the code, Stripe, this is the very thing your report damns as waste.
To maintain software, we refactor, we add tests, we discuss and debate, we tease apart and reconstruct. To the software developer, maintenance is normal and expected. It’s part of the job. You build it, then you support and maintain it.
The reason I’m so hot about this is that I know (from watching it happen again and again) how many executive level managers will interpret your report. They will think it’s because their programmers are lazy, feckless, indolent, and narcissistic.
It’s cognitively and politically much easier to blame bad code on an engineer’s attitude than the ecosystem in which they work. And these executive level managers will vent their frustration on those same engineers and look for quick wins like off-shoring.
"If we’re going to suffer from bad code, at least let’s get it cheaply."
Or the exec might crank up their attitude of command-and-control to bring those apparently miscreant coders to order. Misery for the coder, and never works out well.
Stripe, your report does not help us get better. You’re throwing fuel on the fire.
(Originally posted most of this Twitter, wanted it here to increase the chance of someone relevant at Stripe seeing it)
"leveraging developer talent" : exploit labor without competitive pay
"Access to talent" : noncompetitive pay
"Access to software engineers" : noncompetitive pay
"Access to capital" : noncompetitive pay
"Immigration requirements" : please let us import more labor, but still keep them indentured to us
Not really. A points based system or even something that just means that H-1Bs are instantly given green cards would work for me and I'm not even CEO.
In fact, the current annoyance is this bloody work stoppage going on regarding H-1B premium processing. It's a nightmare.