Software is breaking down at a systemic level
breck-mckye.com
breck-mckye.com
1. Consolidating internet-dependent projects to a few winning cloud vendors and a dozen winning SaaS providers, made all of those project have coincident failure points. Which is great for corporate accountability ("it's Amazon's issue, not ours!") but bad for end users ("shit, EVERYTHING is down today")
2. The movement to migrate fat local-first projects towards thin internet-dependent or server-hosted ones made almost all software share those same coincident failure paths. This was good for entrenching a subscription business model and brought some end user conveniences, but meant everything being down meant everything being down.
3. Anxious and lazy dependency-first development culture ("you could never write that as well as some github rando") meant projects now had an order of magnitude more contributors but only reviewed and owned code for a surface level, making all these projects extremely vulnerable to supply chain failures and subject to constant maintenance churn.
4. Employment and team tenure continually fell so that most projects no longer have experienced owners and instead just temporary custodians.
You can find the seeds of many those problems in the 1990's and earlier and you can even argue that some of them were inevitable (and even worthwhile) destinations for the industry as it evolved without an engineering guild to slow it down and insist on rigor or evidenced safety.
But anybody who's been around for a few decades can attest to how these risky patterns rapidly grew into norms over the past 20 years and shifted the industry from one where any given software had some idiosyncratic bugs that users could learn to work around to one where all software had lots of bugs, many of them correlated across apps, whete failures would wash through and shut down (or make vulnerable) whole industries for hours or days, with effecrively nobody in particular to hold to account for any of it.
Software has never been in a pristine "golden age", but the last 20 years built almost everything on one giant, shared house of cards that's now teetering on the brink of collapse. Better or wose, it's indisputably different than it was before.
I think this is the completely wrong takeaway.
Ober the last ~15 years I have probably used around ~10 web frameworks/libraries
_not a single one was even close to being complete or mostly correct, even just for standard HTTP ~1.1 handling_!!! (The routing, path, header handling. Not any of the complex stuff in HTTP2, Quick etc.)
the problem isn't using dependencies, that is the solution (not reinventing the wheel again).
And preferring reusing solutions which are more complete, practical testet and have more resources to maintain/improve basic functionality is (in most situations) not "anxious" development, but (should be) common sense.
The amount of hidden complexity in even somewhat basic stuff like HTTP is enourmous, rewriting everything all the time is wasting billions across the industry all the time and (in general) increases the chance for bugs.
The problem isn't "using dependencies" or small dependencies, but:
- availability (or lack) of high quality well maintained dependencies
- "no one" wanting to pay for software dependencies
- misguided tendencies for 'clever' & overengineered solution in one extreme (e.g. the stuff which lead to log4shell)
- misguided obsession with deceptive simplicity in artificial examples which lead to 1001 incomplete solutions because of the obsessiv simplicity doesn't stop at cutting out over-engineering but also cuts out fundamental parts
- lack of widely adopted, effective, tooling & infrastructure to trust dependencies.
- lack of tooling to more efficient reuse and compose working parts, the way we do programming seems just very fundamental broken.
- also a lot of legacy tech we are stuck with making many things unnecessary complicated
Also as a side note, number of packages != number of entities to trust. Weather you use 10 or 1 package from the same source, maintained and published in the same way often makes little to no difference for supply chain security.
Take CSV, there are multiple RFCs that fully or slightly cover ab out the standard if you want to call it that.
and some of the things listed are fundamental software issues you _don't_ avoid just by reinventing the wheel, they just change how they appear
honestly in my experience, for most things, problems with reinventing the wheel are way long term larger then problems with re-using software, at least if the re-using is done responsible
I have seen way to many cases of companies trying to avoid using dependencies for something "because this thing is simple" and then realized they are mishandling this, had an outage due to that, had to add those features etc. etc.
In then end often even just taking the 3rd party dependency, giving it a short review and then vendoring it, with rare updates to it, would nearly always have lead to a far better outcome.
(Which is also the recommendation I give for guidelines for "not-too-large 3rd party leaf dependencies", given them a small review, vendor them, keep an eye out(1) for CVEs and then only update when needed with another small review. Through note that this doesn't work for all things, the "not-too-large leaf dependencies" part matters. For other things other guidelines are better.)
(1): As part of normal CVE/Security scanning.
> lack of widely adopted, effective, tooling & infrastructure to trust dependencies
How would I as an individual developer, or even a CTO at a small to midsize company, justify the investment to fix that? It would take buy-in from a Google-level entity to create and rollout such tooling for, say, Java.
Copying a few functions, or even recreating a whole HTTP server from scratch, is a smaller body of work. Sure, I could vendor and keep an eye out for CVEs, that’s a valid approach. But that’s me copying and taking ownership of an external codebase, not taking a dependency. Ownership is easiest for code that you’ve written, because you (or your team) also built the mental model to go along with it.
And if I have to vet a new codebase anyway, I’d rather it be the smallest subset that I actually use; not the full functionality of NGINX.
I can justify making the locally optimal decision (reimplement some functionality so I can truly own it) and that may result in a less than optimal global outcome. I can’t justify doing to the globally optimal thing at the expense of my company.
Ultimately, the incentives aren’t aligned. The activation energy to get from locally optimal to global optimal is too high, and that’s not going to change any time soon. In fact, it’s rapidly trending the other direction. AI makes it easy to reimplement the functionality you need. An idealized view of “if only people paid for high-quality dependencies this would be solved” is not one I can buy in to. That’s not the reality I live in, for better or worse. In a world where nobody wants to shoulder the burden of vetting dependencies, avoiding them is often the best alternative.
I’ll die on this hill. You should pin your dependencies so you don’t get pwned, and then use dependencies for everything mildly complicated that isn’t core to your product.
I bet 90% of people reading this work somewhere that has built a model router in the last year.
I understand the sentiment. You work at a sparkling water company, and building a durable workflow engine is much more interesting than building a website.
But without that tech being your core product, maintaining and evolving it will lose priority. If it requires barely no maintenance, it’s even worse, because when it inevitably does, you don’t know if deploying a new version still works.
Use stuff that exists. If the dependencies don’t do what you need, fork them, and consider contributing a patch.
And if you’re allowing patches via SemVer, you’re relying on authors using SemVer correctly, which they won’t. For example, the common practice of ~never having a 1.0 so the minor release is basically the major and the patch is effectively the minor.
With floating dependencies you still need to track them all to know when you need a new deploy just for the version bumps anyway.
You might subscribe to the dependency but that almost never means you find out about transitive dependencies. DO any of these services also tell you about transitive dependencies ?
Eventually software is going to get regulated and all you HNers will have a fit, but the way things are going now isn't working. The internet is a gigantic security dumpster fire.
3/5
* Resilient
* Cheap
Pick 2
With the advent of individual computers/disks/networking equipment not breaking as much, we optimised for Efficient+Cheap software. This results in systems prone to cascading failures.
The most resilient systems I've worked (mostly telecom) on we built from crappy parts which broke all the time, and they sure weren't cheap.
To me this is the crux of the problem. The issue is not that a particular company’s services went down, but that 25% of the world was relying on it being up.
Decisions are made on the basis of trend chasing, what’s fun, and what’s good for the devs’ resume. And it’s completely accepted, hence the world is overflowing with Rube-Goldberg machines that basically defrauded whoever paid for it. Try doing that as a carpenter and see how long you’d stay in business.
This is a common take, but it's most attractive for its cynicism, it doesn't fully represent the reality.
Cloud vendors are also great for allowing corporations to implement systems that would be prohibitive for them to implement, operate, and maintain as reliably on their own. Even big enterprises often struggle to implement everything they need, that a cloud provider can provide out of the box.
I worked at a multi-billion $ subsidiary of a global Fortune 500 company that tried, as part of its move off mainframes, to "go its own way", with their own hardware in Equinix data centers. The entire executive level of the IT team responsible for that effort - about five or six people - were eventually fired, along with much acrimony, because they couldn't pull it off. There were plenty of warning signs, but the final straw came when there was a major security breach, which was unacceptable because this company had federal customers and were subject to all sort of federal regulations, including CFIUS because the parent company was non-US. (They once had to rewrite an entire major system from scratch because of a breach of CFIUS regulations that made the government consider their code to be potentially compromised.)
That may be an unusual case, but the point is that if even large, well-funded, established companies can get a lot of benefit from cloud systems, that goes double or more for smaller companies, companies in less attractive industries, etc. etc.
(Pointing this out sometimes triggers people who maintain servers themselves, especially here on HN, but the point is those people don't work at the companies in question, and essentially don't scale - i.e. it's difficult to recruit enough of them and manage them and put together a durable system with enterprise-grade reliability that depends on having 10x sysadmins.)
And yes, "shit, EVERYTHING is down today" is a consequence of all this, but the alternative is generally much worse for individual companies.
My friend runs an OSS project with 10s of thousands of users on a fairly realtime and latency sensitive application and it works great and has been very reliable. He does storage, compute, networking, all of it.
I run a FinTech startup on bare metal and do it for 1/10th the price of AWS (In two Geographic Regions). Reliability has been 4-5 9s over the last year. I’ve also worked for a few F500s. I’ve been the guy responsible for buying, racking, networking, and coding. I’ve also worked at AWS and am quite familiar with the pros and cons and nothing about my opinion has really changed, the cloud is a money printer that serves to make certain people extremely rich. Companies and individuals standing up their own hardware and learning how it works is a threat to their business model.
This point never occurred to me but it explains so much. In addition to the points made in this thread, I have seen a dismantling of tech education. For example, I created the two-year degree program for Florida in Linux system administration 20+ years ago. I later learned the program's requirements were gutted by a committee; the took a Windows administration degree document and s/Windows|Microsoft/Linux/g. The new requirements included adding users using Linux active directory and other absurdities. Programming degree knowledge requirements were also dumbed down.
All this because educated tech workers are a threat to someone's bottom line.
You're thinking of this from the perspective of your own personal expertise, not from a business perspective. The problem is that you don't scale. There are millions of businesses using cloud services. You're not working for any of them. Do you want to?
And are there millions of people with your experience and skills? Doubtful. (If there were, you'd be a lot less valuable.) And a company of any significant size can't rely on one person to operate their systems, so you need a team, and you need to manage that team, and the managers have to be good as well.
Also, 10s of thousands of users is tiny in the modern world. Instagram scaled to 30 million users on AWS with something like 13 employees, before it was acquired by Facebook. It would have been much different if they'd had to do that on their own hardware. They're an outlier, but many business are able to use the cloud to scale to levels that would require a massive internal effort otherwise, especially if they have to meet compliance requirements, integrate with other enterprises, etc.
> Companies and individuals standing up their own hardware and learning how it works is a threat to their business model.
They don't care much about this "threat", because executives look at the situation and choose cloud because it makes business sense.
I mean I kept servers online for years, but they didn't see anything like the traffic and security threats that show up now.
"Before the cloud was trendy", small to medium businesses didn't have millions of customers using their online systems.
> ...
> And yes, "shit, EVERYTHING is down today" is a consequence of all this, but the alternative is generally much worse for individual companies.
IIRC, AWS and cloud infrastructure didn't get big until the mid-2010s. Fortune 500 companies (and everyone else) were able to "go their own way" with their own hardware before that, they had to. That was just ten years ago.
If "[going your] own way", with [your] own hardware" is too hard to do now, I'd guess that's not because it's impossible, but because there's been a combination of skill/capability loss (similar to what's been seeing in manufacturing) and places like AWS sucking up much of the remaining talent.
The biggest problem most businesses started running into with server rooms was supplying enough power and cooling. You're starting to talk about running out of power coming in and places for generators where businesses had their server rooms. Instead of new buildings they clouded it out.
what a wild ride it has been. in 1990 there were maybe 500k software / programmers working. today it is probably 20 Millon. at the end of the day, i think this is our problem. too much growth, too fast. to do it the right way.
The PHP2Shell bug existed more than 15 years ago, spanning releases as early as 5.x. There was no AI coding back then. So it's very hard to blame AI. There was a coding mistake, and AI found it in 2026. Probably a good thing before a malicious party does the same.
OSS definitely flourished way before 10-15 years ago. I'm not sure what the author suggests as "knowing the details". Should we all rebuild our own web scripting language to know "all the details": e.g. PHP? Probably not practical. A line has to be drawn on where your knowledge and expertise stop. THe line can be vague and consistently changing based on needs. But there will always be a line. We will always stand on shoulders of of our predecessors one way or another
I understand the author's trying to say the overall situation is unstable and trending negative but I am not sure the writing's logical flow is holding up very well.
A better way to put it is we've written a lot of code, and a lot of it had bugs hiding in plain sight. AI made the whole bill come due at once. And will this AI code explosion create another bill we won't know about until 20 years later? Who knows.
Of course the shaky foundations were there before AI. And before the widespread proliferation of NPM packages etc... But those increase the ease and pace at which we can build on top of the shaky foundations, they increase the rate of so-called decoherence. And there are economic and organizational incentives to keep increasing that ease and pace. That makes it more and more likely that a problem in one of the shaky foundations (whether AI-created or not) causes a system-wide failure, and at the same time results in a more and more generalist tech workforce, making it harder and harder for an organization to respond appropriately when that happens:
> I believe that tech companies risk losing their “organisational cognition” - their ability to think and respond as a coherent unit.
> This is the real threat: a scenario where nothing in the system is done with intent, and nobody in the system has control. We generate complexity faster than we can tame it.
Probably won't change much until the cost/losses to society become unbearable.
In every industry that was unregulated people eventually get tired of the monetary loss, outages, dangers, and deaths. 'The people' force the government to lay down laws because they get tired of the libertarians having man eating bears wandering the streets. Said libertarians will complain for 50 years on why this is terrible, but couldn't be arsed to fix the problem in the first place.
AI is basically at this point a super auditor that doesnt fall asleep.
And to your point the only ones were seeing consistently are the massive dumps of stuff from the whitehats. I am pretty sure the blackhats are doing the exact same thing and just keeping it to themselves to sell.
We are going to get all of the tech debt. All at once. Even the tech debt we didnt even know existed until a day ago.
But now? I member someone on Twitter claiming a reproduction cost them less than a dollar worth of tokens.
That isn't just tech debt coming due, it's a bunch of mobsters armed with the equivalent of gatling guns. But unlike powerful guns that governments can somewhat regulate, there just is no way to get the AI genie back into the bottle. People run abliterated models on their own hardware now, the equivalent of 3d printed guns, just so much more powerful.
I find the arguments about decoherence and the sort of snowball-of-complexity more compelling than this timeline.
Reminded me of this tweet: “This has been talked about extensively you’re just 21”
Just because it is/was new to you at that time, doesn’t mean it is New.
AI is great at following patterns. AI is great at writing tests. AI is great at doing work that humans would consider drudgery and then applying "enough" judgement to it.
The problem is that most software teams and most engineers are very, very bad at the thing that AI needs to succeed: technical writing. Documenting the patterns, the rules, the heuristics, the SOP, the judgement; developers could barely be bothered to write comments before (which in the AI age are free, infrastructure-less memory that is always "line-of-sight" for the agent and can be used to enrich with keywords).
It's not that AI is the problem, it's that we humans are good at resolving these constraints in the real world at a human pace. AI makes that pace impossible for a human to keep up as a trailing activity. Rather teams need to invest more in technical writing and producing the corpus that AI relies on to learn the operational constraints and rules.
As a result I get:
1) The enmity of my lead dev who prefers to do things more fast and loose.
2) Not promoted, due to the first reason and due to the fact that I don't have "impactful" metrics.
I'm not saying I'm perfect, there are definitely things I could have done better. But the fact still remains that the industry grossly undervalues maintenance and optimization that doesn't lead to easily measurable numbers that impact the bottom line.
That in turn is because it's hard to tell if SWEs are actually paying down critical debt or just wasting time. But that's the core problem, not that we're not good at it.
The good news is I can get AI agents to do this kind of work so I can now chase after metrics.
Typically these places are associated with compliance, audits and regulation with teeth. It might be adjacent to lawyers, in public institutions or weird consultancies. At least that's where I've found such work. Insurance and banking might be decent bets, and they sometimes also come with oddball tech stacks, perhaps mainframes, perhaps Smalltalk.
I actually happen to work on a product where reliability _is_ our #1 most critical thing, and even then in certain features and certain situations it is perfectly reasonable to prefer time-to-market over quality as long as we are clear about that to our users.
So, I'd encourage you to think from a product perspective.
This is possibly due to the high level of economic inequality, meaning less resources in the middle of the economy.
In a decade I think we will have few people who can competently write or even read computer code without AI assistance.. and I think we might say the same about math and science and medicine.
AI assisted learning could lead to a renaissance of human capability .. but thats not where we are headed, we are outsourcing all the hard thinking skills. In future, only a few crazy retro nerds will learn assembler or multivariable calculus, just as only a few crazy nerds now make mechanical watches as art objects.
Excessive inequality means we wont be training young scientists, and young couples will defer having children until its biologically too late.
High inequality + LLM AIs + climate change is not a recipe for human flourishing.
Being good at one of the trades however? That's a pretty safe bet. It will be quite the time until physical robot "bodies" are capable of the dexterity, size and endurance of a human. And by the time robots get that capability, we're either living in a Star Trek utopia or in Matrix style neo-feudalism.
Friend, I work for Google. The high level of inequality works completely in our favor, yet we are incentived to churn out slop that hits metrics.
That is not true. Microsoft is extremely profitable. Apple as well. Amazon. Banks are literally printing money. All of them have resources to produce robust secure software.
They just decided not to go that way.
Regulations are written in blood. No industry regulates itself at first. It's always no brakes, no safeties until a horrific event occurs, then the crackdown comes.
Which makes sense: those in a position to promote others generally don't understand the technical part. What they see is how fast they get a new toy to play with. And if it breaks too soon, they will still be able to blame the developers.
Maybe part of it is laziness but for sure part of is low morale and hopelessness.
That's fundamentally impossible though -- if the information isn't there, the model is just going to move toward the average and fill in the blanks.
On another note, I'm also not sure where I stand yet on the thing I see where software devs end up falling down the rabbit hole of building little mini software factories, with agents reviewing agents, coordinating agents and having agents cosplay human roles -- essentially 'orchestration porn'. I'm yet to dive into that rabbit hole (I likely will at some point, so I'm not judging), but from the outside it looks like you end up worsening your understanding of the system, while spending tokens unnecessarily and creating a closed-loop mini model collapse.
I've actually run into the situation at work where I've asked people whether the libraries they've made can do a particular thing and sometimes the answer has been "I don't know" (I'm talking just _features_, not even low-level architecture). At that point, something has gone wrong.
I'm starting to get really interested in "Okay, if human code review is fundamentally impossible given the volume of code, what can we do?". We have a tool that can essentially build tools for almost free, so I'm trying to approach it from that angle. Let's see what sticks.
It also creates an expectation that anything done with AI as part of the process was done in one shot, and therefore the person had no meaningful impact on the output.
I saw a Reddit post of someone asking essentially “I’ve spent the last 7-8 months working with Claude on a game, am I really making something?” and I just wanted to reply “of course you f****ng are, don’t let people tell you otherwise!”
Write less code. Reuse things that have already been written and thus were already reviewed and tested by someone else. Do not reinvent the wheel. KISS.
With AI everyone is implemented a dozen libraries from scratch in their project instead of using something someone else made. Why use a library for 2D geometry when ChatGPT can write one in 10 minutes?
More tokens, more money.
Software engineers were at least pretty good at technical writing using the artificial languages we created to enable clear, unambiguous technical writing... programming languages. Seems like software engineers might do better using AI if they remembered this and started using pseudo-code as their language for spec writing (as many have recommended).
They can even write good property based tests, with impressive Arbitrary instances for everything!
The fundamental problem is that current AI tools don't learn. They make the same mistakes over and over again, unless you give them explicit instructions to avoid those mistakes. Maybe we need to accept that all work is now done by new hires, who have read all the documentation but have no experience in anything. Or maybe AI tools will get better, and all that tedious documentation effort was just a waste of time.
and
>and all that tedious documentation effort was just a waste of time.
Seeing where the AI is failing and then putting out good public documentation is a good way to ensure the next released model learns.
I've been one of the organizers of a relatively large RPG convention for ~25 years. If I had to write down all I know about that convention, it would easily take a year or two. Because most of that knowledge is not in a form that can be easily expressed in words. And in my day job as a scientist, a similar task would easily take 5–10 years.
> The fundamental problem is that current AI tools don't learn
They don't learn, but they are very, very good at searching. That is why writing things down is important, but so is getting that into some sort of agent accessible index.Where do you get the correct test results? First from the specifications, then gradually from the edge cases found by testers and eventually users. It is not one shot process it is a big loop which will get repeated over and over again until users will run out of things to break.
The software is breaking because nobody is bothered to write down what is software actually supposed to do, and from there to create a set of tests which will anchor the behavior of software.
So who owns outcomes? Usability across the board? General customer experience? Nobody. The actual software that people pay for, is orphaned, and anyone trying to track down problems is just asking to be blamed for them. Thus, most of what happens is undefined behavior, like if we were writing an old C standard. Having someone that has the responsibility to make the product as a whole do something, and thus maybe be able to write the spec eventually, would already be an improvement.
The same rules that you, an expert would apply, should be documented and used by an agent to apply those heuristics.
There are obviously some areas where there's spot judgement, deviations, and exceptions. But by and large, you have some heuristic that you use when evaluating output. Document it and give it to an agent.
But I wonder: am I possibly engaging in wishful thinking? I believe myself to be good at technical writing, so it would certainly be very flattering for my ego if technical writing were suddenly the thing that AI needs to succeed. So I'm a little skeptical of this perception of mine!
How would you convince someone skeptical that technical writing is the thing AI needs to succeed?
(QA, incidentally, is another thing I think I'm particularly good at, and I have the impression that it's also crucial to the success of current AI, and it's not the same thing as technical writing. AI can identify when code crashes, doesn't pass automated tests, or doesn't look like the wireframe, but it's not yet very good at identifying when it's difficult to use or confusing.)
I agree that humans are the problem and their gullibility towards token prediction algorithms and CEOs blathering.
By people lets start with management, if you dont know how to program then you really dont know all the layers and all the stuff that does or doesnt happen or what can or cant happen.
For most people software is just magic.
Many so called software engineers dont really know how basic stuff REALLY works. How can they counter or improve something if they dont understand how osmething works or can be abused, and tahts only the start.
Plus your documentation can often work very well alongside whatever skills you have created over time, because you basically have two sources it is drawing on telling it to do the same thing. I find it leads to less off book behavior.
I’ve always been perplexed by how people see AI as this creative tool or something to innovate with. That’s not how I see it at all. I see it as a device that can follow a template very well if you lay good tracks. They can archive and check hundreds of pieces of media way faster than I can. But I would never trust it to craft a story
I consider the comments AI writes to be far worse than the code. I am rewriting them a lot more often than I rewrite the actual code. They are overly wordy and include a lot of unnecessary asides that make it sound like a LinkedIn post. Or it will reference specific irrelevant things in the prompt that won’t matter when someone looks at it 6 months from now.
Things that are really useful in comments:
- URLS to llms.txt or other authoritative docs so agents see it, read it
- Related entities that may require multiple hops to deduce; you can short circuit the agents read by directly pointing your comment to a related entity that may not show up directly in the call path (e.g. separated by a message queue)
- Reasoning and "why" a decision was made so in a future pass, an agent understands why a condition or rule exists
You can write bad comments, but you can now just prompt/instruct your agents to write good comments and use it as long-term memory.4/5
It was like that before AI, it won't get better now that we need even more people to pretend they know how to manage developers.
I've seen it first hand. 70 people fired, 20 people left. The 20 that are left take on all the work that was done before, plus some extra, because every company now has a new AI product, obviously. AI agents make us so productive. Every review is a 30 second LGTM on 500 new lines of code. Confluence filling up with AI slop.
A few weeks back I saw, first hand, a DevOps lead getting ready to ship a big update to performance-critical product where Opus 5.5 suggested they completely turn off JIT everywhere because they were running into a HotSpot C2 optimizer bug that would make the first request after a restart fail with a 500. Actual fix was a one-line adjustment somewhere in a library.
People are scared of being replaced, so they need to appear as productive as possible.
A lot of devs still think their job is to produce code; it's actually to produce all of the guidance and structural scaffolding for the agents ("the harness") to guide its behavior and output.
A concrete example: I wrote a small library[0] (MIT, OSS) that improves how agents write unit and integration tests. This isn't product code, but this makes the agent better at writing the tests it uses to verify its work. (Tests become denser, more semantic, less context, type-checked setup, etc.). You can't expect an agent to do this by itself; the human needs to produce the scaffolding.
If its ends up being really that straightforward, it sounds less like engineering a product and uncomfortably close to building your own to corporate-owned replacement so that you can be fired.
This basic reasoning is wrong and leads to “orchestration porn”. Thinking that you can laid down rules to automate software production like it’s a factory pumping out products.
What you forget is that software are grown iteratively, not guided down some assembly line. Every patch is a revision of what the software should do, not the next step of a workflow.
So stuff like patterns, architecture, tests,… are helpers for communicating the purpose of the software. They’re not really relevant to the software itself as the computer would be happy with just a single file.
You have a software that’s needed to fulfill some needs, you have a team of people where each brings expertise to build it. Workflows are built so that everyone how to contribute, what they can depends on, and what others need. When there’s a failure, we add the missing information. When there’s need to do some changes, we inform everyone impacted.
Everything is information and communication related. Information about the system and communication between everyone that interacts with the latter.
So devs do not think their job is to produce code. It is to take requirements written in ambiguous natural language, sometimes incomplete and contradictory, and to design a solution that is formally written while taking care that all statements are coherent.
Making things coherent (aka correct) is what takes the majority of our time. The other is gathering requirements. The actual typing of code is pretty much instinctive, just like you don’t check the orthography of words like “concrete” or “guide”, and rely on the spellchecker for more difficult one.
The “orchestration porn” part is that there’s no mention anywhere on how you think correctness and be improved or even maintained as part of your tool.
> there’s no mention anywhere on how you think correctness and be improved or even maintained as part of your tool.
The first paragraph is literally a list of examples, followed up by a specific library that helped the OP. The paragraph you quoted explains where the maintenance lies.
AI agents don’t have this in a sense; or they do, but it’s average and stochastic. We have to be in the loop to provide judgement and manage the agent’s context; the content of that context steers the agent in the direction that our judgement tells us the development should go, to meet the various requirements. Some of those requirements are around the software’s function, some around maintainability, some around understandability, etc.
I think scaffolding is good at decreasing the amount of repeated mistakes and ensuring the agent can request and receive information efficiently. I think software factories are too self-referential; I’m not set on that opinion though and like I’ve said in another comment, I will fall down this rabbit hole at some point just to see what it’s like.
So many times I’ve wondered “you know how to do this, so why didn’t you just do it the first time?” — the old “Oh you’re absolutely right! That’s a much better way to do it!”.
It’s not that the model doesn’t have the knowledge, it’s that it doesn’t access it at the time it is needed. Part of my job now is to collaborate with the models and the tools.
The first is solved by deterministic tool as code is already formal, so the grammar is very strict and fully defined. But the second part is entirely human dependent of what the end result should look like, even when the same semantics is being kept. That's why two people asked to implement the same task will code it differently even if from the point view of the machine, they're equivalent.
Maintainability and readability is not about software correctness. It's about how the human mind can absorb what the software should do, aka the specs embedded in the code, which is just formal notation. If it's easy, then the software correctness is easy to check and validate. And when the requirements changes, it's easy to apply the needed modifications.
So asking agents to improve upon that is like asking an agent to write an email. It's an alienation of human-human communication.
i have written alot of this type of documentation as an architect for human teams, and i find it works for agents as well.
Building "orchestration" without being able to deterministically measure improvements is not engineering, it's cargo cult.
I also have a set of tools[0] specifically built to address this by surfacing agent usage of context and harness usage of tools.
Do I version? In some cases, yes. The tool I built for our team has configurable code review agents that trigger both in PRs and "in-the-loop" for the agent (exposes the same remote reviewer as MCP). It has an option to build a dataset to backtest changes to the code review prompts[1] and prompt history is saved as are dataset versions.
[0] https://zeeq.ai/
[1] https://zeeq.ai/docs/managing-zeeq/agent-code-reviews#backte...
Deleting first is what changed it for me. I go through the suite, cut everything that doesn't map to a flow a user actually goes through, then keep a few that drive the deployed app end to end. Fewer tests, noisier when one fails, and I trust them now.
The test plan is still the bit I can't hand over, because deciding what a flow is supposed to do is where I get it wrong myself.
You have to stop thinking this way. It's our job to do what our management wants us to do, which is often to produce as much software at the lowest cost possible.
There are legitimate criticisms about AI use and how AI has changed the profession, but you have to learn which battles are worth fighting.
The refusal to take responsibility for the consequences of one's work is one part of a vast complex of machines that normalize the horrific. We have met the shoggoths and they are us.
The IT world needs to internalise understanding of the Peter Principle and Hanlon's Razor.
Don't just do what you're told. You also need to know why you were told it, and when to creatively disobey.
Embrace the power of "no".
Have you ever been in a situation where you did what the management wanted you to do for years, regularly warning them that you could not write good code while doing what they wanted but being dismissed because they knew better? And have you ever been in a situation where you finally got your "I told you so" moment after years, when they actually realise it won't work?
I have. You know what happens then? It's not their fault. Best case, they explain why you couldn't have known better because it was hard. Worst case, they fire you because... well you built that mess, remember?
What's the winning strategy for a dev in that situation? IMO it's adversarial: do whatever works for you: work less and pretend it's hard, or chase a better salary, don't get attached, and prepare to jump ship whenever needed.
Is it sad? Yes, but they start it.
Personally, I was asking "but what's the use case" in every meeting. "Why would somebody pay for this". "Who's the customer". Right now the vibes are quite similar to back then.
Damn if only there was a way to write strict rules and patterns in the english language that could be converted into strict instructions for the computers and ways to verify those instructions...
Pathfinder lets you understand your product’s building blocks. So you and your team pilot your agents at the product level, not the code level.
It’s augmented product engineering: https://pathfinder.com.ai
I feel this piece is either more alarmist than it deserves to be, or it is late to the party.
It uses various incidents in 2026 to prove a point while dismissing those that did take place in the decade or so previous. For example, heartbleed, the log4j RCE, Rails' mass assignment DOS, Cloudflare updating a WAF rule and breaking the internet, Atlassian deleting almost 1000 of its customers' accounts and data, the Ashley Madison breach, Crowdstrike bricking their customers computers, Okta getting owned multiple times, etc.
My own point is that software became like this when we started becoming pragmatic. Nothing to do with AI or management really, we just realised, especially with web tech, that you can release something and then fix it. In the old age of getting CDs or floppies printed and distributed, that was not even remotely feasible. A similar incident in the 90s would involve a product recall or a way to get updated media from a retailer with the fix. Very expensive.
Standards have dropped and continue to do so, because we can also fix fast. It's not suddenly breaking down. If anything, it's just too centralised.
The harsh reality is that some people/systems are adapting just fine to the new way of doing things, others are making a mess of things by rushing via vibe coding and not checking the work (either by choice or by circumstance at work). The results won't be (and shouldn't be expected to be) linear.
The 2010s brought on a very groupthinky dynamic to software dev. This led to a culture where the majority of developers just ask "what is everyone else doing." Now that we're past the blog era of development, that shared reality appears to be breaking down. Now the answer really is "whatever makes sense for/to you," but I think (anecdotally) that's freaking a lot of people and teams out.
For the first time in a long time, a "best practice" doesn't really exist.
Now, almost all of our competitors and us have taken a fundamentally different approach to how we use the AI tools for our work, and it's been really fascinating and fun to compare and contrast with them. We're finally different, and to me that's a great thing.
I'm sure approaches will converge into "best practices" again but for now I'm really enjoying that we don't all have to be the same.
This is making teaching software engineering really fun, I tell ya... :)
And it’s changing so fast. A year ago it was still best practice to read and understand all AI code.. but that has rapidly degraded and as much disputed. Now somewhat argue you don’t need to review the code if you have multiple AI code review persona’s reviewing it for you, but you still need to understand the change, etc. etc. it will take time for this to settle out.
I'm not sure if you're making excuses for those systems that are failure but I'd say no one should have an excuse for these failure - whether it's "there are no best practices" or otherwise. There were best practices previously and until someone can be sure what they're doing is good, they should be following those old approaches.
It's a bit like someone who says "everyone's doing coke now and some of them seem to be OK, that means I should be able to do it too 'til I get the hang of it".
Not necessarily, just acknowledging them as an unavoidable reality.
Claiming something is unavoidable when it is essentially just hard to avoid and most people don't avoid it, is essentially making excuses imo.
The actual programming logic has certainly always been complex, but the integrations weren't so bad. You could learn your very small toolset quickly and focus on ideas. Now, you have an ultra-massive toolchain to come to terms with, made up of free software and "cloud" architecture that starts off cheap and gets expensive later on.
Dependencies have always been the problem, but that problem has expanded in every direction, at every layer, approaching our own limits of comprehension. AI has allowed us to catch up, temporarily, but given the collective penchant for upping the ante, I expect the real supernova is yet to come.
Part of it is the inclination of the developer to make the thing more robust. Part of it is lowering the barrier of entry, maybe to the point it's _too_ easy to submit a package. Part of it is bad decisions made by the package manager, like running arbitrary scripts.
Java via maven central is similar. It’s not a popular language. We know those libraries are riddled with issues - log4j’a big vulnerability isn’t that old - and simply requiring a GPG signature on a package is a thin guard against compromised uploads.
The NPM ecosystem is uniquely suited to supply chain attacks though. It has a massive rate of churn, which makes it impossible to maintain dependencies safely. You either lock in and hope there’s no big issues or ride the wave and hope you don’t get hit by zero days.
5/5, account is now locked
AI has allowed us to scale development effort massively - and shows the same weaknesses as scaling human teams too quickly - planning, verification/testing and code fragmentation become issues almost immediately.
In the past few years though, it’s been creeping into my domain too.
Even this needs a qualifier. Radiation exposure can cause bit flips. Certain aerospace applications must address this, and even some earth-bound applications at high altitude.
A quick AI-assisted search informs me that neither AMD nor Intel document any on-CPU error correction, though perhaps there is some. Apple currently has zero systems that even use ECC, which seems insane to me given they’re shipping systems with up to 512 GB of RAM.
This seems to be an issue that’s falling through the cracks again. I guess nothing will be done until there’s a critical failure due to a bit flip, and verifying that was the cause for a given problem is difficult.
On ECC, Apple needs to Think Different (tm). Maybe it should introduce actual “Pro” machines…
Now it’s somehow guarantied that the volume of dirtied quickly prompted version of anything anyone ever cherished far exceed the scarce outcomes of the original.
No matter what's trending worse there's always this thought terminating phantom law of conservation of everything. The fact that something is not an entirely new bespoke phenomenon always turns into denial that it could be happening more often or in a newly qualitatively worse way.
Today, the average code that even a product manager or junior engineer submits is actually excellent overall - if your processes, harness and CI etc are well designed. It is actually super easy to author software well now. Whether you build something that is actually needed and is tasteful is another matter.
99% of the security vulnerabilities being found and exploited today were created by people. Cutting most of them out of the technical details is actually good.
Even if individual code is of higher quality, if there are more pieces of code talking to each other, there are many, many more failure modes because software often fails at its interfaces. Nancy Leveson from MIT has written well on this subject.
Perhaps a bad example, but isolated code will often fail less than networked code that has to interact with a bunch of stuff. An old “dumb” tv never seems to have software issues, my new “smart” tv consistently has issues. It’s like having a dozen people write a paragraph each or a single person write a paper. Even if the single persons writing isn’t the better than everyone else’s, you’ll often get a more coherent paper.
Too many things doing too many things is an apt description for the age we live in.
How many times have we seen file upload libraries allow path traversal to happen ?
Nearly everywhere people dont even attempt to validate taht the effective path from the url is actually under the target directory and have not travelled somewhere else.
WHY
Because most oeople are lazy and dont bother inserting that one line, they just let it go and bad stuff happens.
I think this is the core insight that I keep coming back to: technological advances always entail a transfer of political power.
We would all be better off, when evaluating new technologies and listening to their evangelists, thinking hard about who gains power and who loses power as a result of adopting that technology.
True. But it's not hard to check how control over a new technology is distributed.
A few service providers with centralized control knobs (OpenAI, Anthropic & co), vs mostly decentralized with end-users doing their own thing on their own devices (local-running models, open weights etc).
If we hadn't spent the last decade circle jerking about how we're all 10x devs and everyone else is a pleb there's a chance we could have built unions for ourselves.
That would mean we'd be able to actually wield political power collectively and establish standards for quality and technology adoption like all the other critical industries have.
We don't need to say "we're not adopting tech", we need to say "we're going to follow our process for adopting tech" which may be slightly slower, but the banks would still be online today. Further, it doesn't even necessarily need to be process, it just needs to be something we have a say in instead of a hand full of the craziest narcissists to ever walk the earth.
That's not true at all
I don't expect being in a union would improve my financial or employment situation. But even if it did, I'd oppose it unless there was some specific way in which I felt I was being treated unfairly (there isn't).
The sort of people who think there needs to be a centralized authority that prescribes standards for software are delusional. Software is the part of the economy that has worked the best over the past 20-30 years, and they want to ruin it.
The analogous situation is brewing. The depth and complexity of our computational infrastructure is increasing, and the impact of an incident is less and less likely to be contained solely to the affected system. Eg, a router replacement goes awry, and the entirety of the world’s aviation industry is broken for weeks.
Intentionality isn’t necessary: this is a systemic problem.
The Outlook outage led to 15 people getting chlorine poisoning, for example. Admittedly, that was a horrible setup, but it's an example nonetheless.
SOmeone is paid to do that job but they got lazy and pretend the computer does eveyrthing while they browse tiktok.
When the official cited story (put forth by a known boot-licker) is contested by people on the front lines, I would defer my conclusions until more is known.
One discussion was talking about switch broadcast traffic, which actually kind of passes the smell test for a problem like this. Or at least the switches/routers were blasting traffic at a higher rate than the network could handle.
Of course this sounds like some shit I'd have done in my early years so it's really likely they had some way underskilled people working on this in the hardware and settings and it blew up in their face.
The seed was there since at least a decade, AI just accelerated everything.
> The seed was there since at least a decade
Yes, and the seed was always there, it's intrinsic to the way the economic and market systems are set up.
> Microsoft rushed Azure out of the gates under intense competitive pressure. Corners were cut... Instead of going back to the drawing board to tackle the growing technical debt, Microsoft relied on quick fixes
These aren't technology problems. They are what our broader systems incentivize.
> Because there were fewer details to take care of, tech professionals began to lose their specialisms.
Once it was possible, the market incentivized paying existing employees a little more and cutting out more expensive specialists.
My current working idea is that at a broader systemic level there are issues. So, I agree with parts of this article and your statement.
But the focus on NPM, when things like Ruby gems and other open source tech already existed, feels like picking an arbitrary starting point and makes the rest of the timeline feel a bit off to me. The issues existed earlier than NPM. It also feels like the article is somebody working their way toward understanding. The writer can see common threads, but when they try to sew all the different events together seamlessly, they haven't yet quite gotten to the actual nature of things, and can't quite do so.
> the stock market prioritises short term trends over long term engineering.
Very close--It comes very close to what I've been thinking, but doesn't quite totally line up.
Corner-cutting, cost-cutting, over-reliance on uncompensated work, poor organization, bad QA, people who focus on the short term for a few years then move to another job, etc. all exist in other business sectors. Our systems incentivize these behaviors. The security and stability issues (which aren't new) are the tech version of a car recall, or a Nebraskan town getting poisoned by an ethanol plant.
For tech businesses, "moving fast" and accepting the trade offs is/was a fairly logical conclusion within the rules of these systems. The people on the ground will potentially always feel like they don't have enough of some resource: time, talent, etc. because that's the beast.
With the advent of LLMs/generative AI, several trade offs have quickly changed. Things that were previously somewhat safe to marginalize are suddenly priorities, and obviously organizations are not always good at changing directions quickly. Additionally, new companies are being incentivized to enter the market with the idea thanks to "AI" they can do more with less, but they are often unaware of or actively trying to avoid the the full level of effort required.
Attempting to draw some conclusions, these are two of my next thoughts, which I'm not necessarily looking for direct answers or responses to here:
1. Does the software industry really care enough to change. Some right now are speculating that we're going through a painful phase while we effectively pay down the debit of what used to be in the margins. People are risk avoidant, and before uprooting their methodology, they may endure some pain while waiting to see how things play out. Maybe it's not worth changing and this is actually temporary, or on the larger scale this "crisis" is bit of a tempest in a teapot.
2. As long as approaches like cutting corners are effective, some actor will cut corners. If we decide to change, we probably need to consider changes to the system and/or incentives. This may be things like regulation or ensuring that people can stay focused on the long-term and foundational needs while staying healthy and keeping a roof over their heads. (Any regulation should be careful not to further entrench current winners or bad actors.)
On a personal note: I'm sick of tech companies trying to take control over devices I own and force me to use their deficient programs, broken interfaces, and bad products. It sucks and feels bad. I do not like the end result of these systems.
Some things happened. AI is also happening, therefore AI is to blame (in the way that some people really believe a problem isn't a problem unless you test for it) and here's a lesson about organizational coherence or something.
These essays are the real AI slop of 2026. Not authored by AI, but always trying to get attention by tangentially being related to AI and getting the anxious upvotes of the "can I turn back the clock?" crowd. And there's a humorous commonality to all of these pieces that they all start with some sort of "don't think I'm not with it. I'm with it! I hand slung an LLM off the shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate." call to authority, then pivoting to the "everything has gone to shit" spiel.
We've always had a tenuously connected, barely operated network of systems. We've always had bugs and engineers doing shitty things. Every organization goes through periods of upheaval where all of the institutional knowledge is invalidated or discarded regardless.
None of this is new.
The security issue motivated all Bitrise customers rotating their secrets. this was a big deal for mobile deployments but didn't get much attention on HN..
The problem software engineers are facing is that now that software is ubiquitous, almost everything needs to be 100% reliable (or close to it). This is coming into conflict with the move fast and break stuff everything needs to be done this quarter at minimal cost mentality.
Hence why we are at the breaking point.
At the end of the day regulation will stop the madness, but no one really wants that as it's typically a terrible solution in itself.
The recent shutdown of the US airports is a perfect example of this. The current system is old, with old computers and i believe old analog networks. They worked but one small change installing a routers and some other new systems caused something bad to happen.
In Australia not that long ago, there was a overhead power line that fell and this caused the trains on that line to stop. This then had the ripple effect of causing other lines that feed the initial broken line. Traffic on roads now also drastically became worse because more people were trying to drive or catch buses and all that.
Everything is more complex and connected.
The cloud is a part of the problem, its just another layer that can and does go wrong. Previously when teams deployed to their own servers, they didnt have to also configure the cloud. They were also not a passenger on the nuances and possible failures of the cloud system from its own management technical and otherwise.
Software was never a system. It was always a bunch of twine and duct tape.
It was always brittle. It's not any better or worse than before. You only hear about it more.
Microsoft having global problems now? Microsoft had global problems a decade ago. More things change, more things stay the same.
>I've said for at least the last twenty five years or more that software engineering...isn't.
>It's not "engineering" at all. It's still a craft.
It's about time software engineering is taught inside Engineering school rather than being outsourced to Computer Science.
This one is interesting. Not on its own merit which is more like a mother of facts, prime phenomenon, but that it was mentioned at AI. I had this feeling of decoherence with all the various techniques and approaches that existed out there, from programming languages through data management through composing instructions through user interactions, into the way orchestration of all of these, way before AI. Somehow none were matching the specific composition of products needs completely so had to tape together available parts at loose couplings, hammering things a bit if needed. Or choosing those that fit together perfectly on the cost of giving up on requirements/functionality for the product. Lots of good libraries not good enough for some details so a new one emerged or the existing one tried to be so many things that it reached digital schizophrenia stemming in complexity (allowing flame or even religious wars between developers about best/sacred approaches/techniques/mentality/...).
With AI I feel different: the promise of purpose made things fitting together well.
I think only one of those is true. It takes a lot of effort to get quality architecture and not-function-soup code (at least in Python) from LLMs. I can't imagine the pile of crap code bases these software factory type solutions are producing. Well, you can see a lot of them on GitHub I guess, so no need to imagine.
Further, one of the negative impacts I've seen from being able to go a lot of faster is that it translates to product teams being able to ship faster which is not always better. We've lost the "back pressure" we used to have in software development which forced product teams to consider, then consider again, then consider again as each new sprint requires reevaluating priorities. This resulted in a lot of requests stalling out and never being built that should have never been suggested in the first place. Or, to be more charitable, the delay gave the product team time to get over the initial excitement of a new idea and really consider if it was warranted.
Now we are building all-the-things and I think that's only better in some cases. I think it's proving worse in most other cases. Especially when you consider that every line of code that ships has liability attached to it.
That results in large bloated mess projects that no human can reasonably understand. That means we are dependent on LLMs to understand it and, the larger the code base, the more LLM tokens are required and the worse the LLMs are at understanding all the relevant parts and making wise architecture decisions. This creates a downward spiral that is harder and harder to manage and means features take longer and longer to ship (although, still maybe less time than pre-LLM dev...we'll see).
Problem is I’ve had 5x more incidents. AI is operating at 10x the speed of a human so even though it’s failing less, it’s causing more incidents per month.
I wonder if people are ready for this kind of paradox where using AI may be safer and faster but less reliable for “human time”.
Perhaps no one is intentionally destroying anything in this case, but malicious compliance and simple negligence can go a long way when it comes to disruption of complex systems. Destroying a sewing machine requires actual violence. An air traffic control system is an abstract concept. You don't need to touch it to break it.
There isn't a software system in the world for which we can't predict the failure modes and make solutions for. You simply ask two questions: 1) "How can I tell for sure this thing is working?", 2) "If it doesn't work, how do I fix it so it does work?". You might not know the answers at first, and that's fine. The business's job is to ask the questions, do all the work to figure out the answers, and then test & implement the answers. If the business does this, they prevent most of the failures, and have a quick deterministic solution for when eventual failure happens. If the business doesn't do these things, they are choosing to let it fail in an unpredictable and likely very expensive way (they don't give a shit).
This process has a bunch of names. Failure Mode and Effects Analysis, Business Continuity Planning, Risk Management, etc. No matter how you slice it, it's the same basic work: find out what could go wrong (at a high level) and have a plan to fix it when it does. It's not always easy, but you can always figure out some kind of plan that works.
For all the security and reliability issues we've seen lately, there are known methods to either prevent those problems or fix them quickly. The incidents aren't happening because something new is going on. They're happening because businesses stopped giving a shit a long time ago, and the chickens are coming home to roost.
100% this. 30+ years in tech, this is always the source of slop. Faster, cheaper, and cheaper again are always bigger drivers than the engineering.
Imagine using AI but still producing the same amount of output/outcome. That would be delightful, great software could be possible… but it’s not “productive”
It was a very long time ago indeed that you needed to manually edit popular programs to build in your environment.
"when tools like NPM hit the scene in 2014, it really felt like a gamechanger"
I can only conclude this was written by a young'un for whom the beginning of software history starts with the JS frameworks.
At least some of these issues are ones that already existed, but are now being discovered by AI.
If that is the case, we would not see the exponential increase that the article mentions, but should instead see something like a temporary blip where discoveries in existing code spike, but eventually all get discovered and fixed.
I would love to see some more data that specifically looks into the number of issues caused by pre-ai code versus post-ai.
That being said, the comments about decoherence strongly resonate with me. Things are moving so fast in so many different directions right now. I don't think we know the right way to build things in this new future yet.
I remember the first time I installed Linux in the early 2000s and was and owned within 5 minutes of putting it on the Internet. Desktop computers of any OS crashed regularly. Servers crashed and needed total rebuilds.
If anything software is way more reliable than it appears because the scale of computers and software is thousands of times larger than it was 20 years ago. More things can fail because there are just more things.
Things were less convenient and generally harder to set up back in the days (we had to fight with XFree86 configurations, desktop environments were immature, a lot of hardware didn't work due to lack of driver support, application software for many tasks were unavailable, there were barely any commercial games), but the kernel had excellent network support and scaled very well across multiple CPUs already (unlike Windows NT's SMP support, let alone the single-CPU-only DOS ancestors). During the 2000s, the Top 500 list of supercomputers was slowly being taken over by Linux clusters, and that wouldn't have happened if Linux would have been unstable. For developers, Linux was (and arguably still is) heaven on earth, and developers actually made sure that "their" system worked well for them. And it did.
There were many difficulties with software in the past, but Linux was rarely one of them.
Absolutely happened.
But then I went on to do Linux hosting and co-location for the next decade.
I don't see why you felt the need to respond. Linux is almost certainly more reliable and secure now than it has been historically. But software continues to grow and get new features, so there will always be new issues. Linux is not a special snowflake in either direction; it's software made by humans.
In my view, however, what is more likely to happen soon is usage of LLM to transform various parts of code into the form with much more clearly defined interfaces/rules/boundaries that can be essentially proven to work correctly. That in turn (in my view) will enable much more robust systems.
This is classic surveillance/detection bias: https://healthjournalism.org/glossary-terms/surveillance-bia...
It’s basically free to black/grey/whitehat penetration test anything and everything without having to even know what ghidra or metasploit so everything gets penetrated
This wasn't always the case, by the way. It started with the "dot-com" mania and VC investment in the late 1990s. Before that, a programming job paid decently well but it was on the level of other cost-center professional jobs such as accounting. A comfortable living, but nothing extravagant.
Networks/internet added a lot of value to people that could program.
The one compelling part is that now we might be forced to pay for some of the rickety towers that we've erected because AI has made exploits pretty cheap to try.
Like before, individuals in an organization had various interests, motivations, connections, knowledge, context, that allowed our organizational brain to function properly. Now, everyone is (or will be in the future) a meat proxy, isolated, and disengaged which produces an organization with neurons but no neural connections: apparently healthy but, as the author put it, decoherent.
This seems a bit myopic. Or naive. Open Source "happened" a lot longer than 10 years ago. Microsoft didn't shit itself about Linux just 10 bare years ago. And it's not like prior to 10 years ago, the narrow focus of npm world wasn't cobbling web sites together with bits and pieces garnered from wherever.
That said...
> But AI is both a cause and a symptom
This I agree with 100%. The rise of AI and necessity of AI to continue to make progress is a massive indictment against an industry that prioritized more features and uptime over solid first principles for years, taking out promisory note after note in complexity, in hopes that the stuff papered over would quietly just fade away.
Preventing the Collapse of Civilization - Jonathan Blow, 2019
https://www.youtube.com/watch?v=pW-SOdj4Kkk
Which focuses on software, but gives many historic parallels of societies forgetting how to do things.
Previously discussed:
Software was never engineering. It’s more like writing a novel, which is way harder because the success criteria is ambiguous.
Now some folks are finally being forced to understand this reality.
A better narrative is that software quality has been very low for some time, and we are lucky that the threat actors have not compromised the assets as broadly as they could have been .
We have a great opportunity with AI to patch up the duct tape and popsicle sticks. Moreover, to realize that consolidating authority into central management with synchronous coordination is terribly brittle.
In other words, we can fix software by dismantling some of the dependencies.
On the other hand software is improving dramatically at a systemic level, even without safe system languages arriving on the horizon.
Empirically, we can observe the breaking effects of this broken thinking. Code quality goes down, and apps go from being 99% to 95% reliable. But that's not a big deal, they say, that's good enough... But when the quality of all the building blocks further down get degraded in the same way, you can't keep track of which parts are broken, and which parts are dependable, and the whole thing comes crumbling down.
People keep saying, "I can build an app with AI, therefore we can build everything with AI." This article talks about some potential anecdotes that could help to show that doesn't work.
The problem for LLMs is less formal and way more wicked. We don't have any good theory of software reliability. At all. All the reliability we have is artisanal and designed ad-hock.
We prove properties of programs all the time, that's what a type system is for.
Can we prove all properties about our program that we're interested in? Maybe, maybe not. But Rice's theorem is not the tool for that argument.
Of course the point is not whether we can now do this with AI, but whether AI can help us get the same quality we used to get, faster.
Also, consider that it should rarely be the goal to enumerate and judge every possible behavior of a program, with exceptions for life-saving or otherwise critical software.
That's called outsourcing, which inevitably enables "vibe" effect of the coding: Your AI goes out of its constraints and does totally irrelevant shit you didn't ask for or does something barely passable you could approve, without knowing further consequences because you outsourced your thinking and control over the project.
To try to find a remedy for this shortcoming, they invented "skills", which totally negates the purpose of an AI (LLMs, genAI, whatever you call it to describe the current tech) IMHO.
It's also funny reading these comments. Most say "well, it was already broken" and they continue trusting AI more. Well, AI is not exempt from this "brokenness".
It reminds me when I first started work on a codebase with proper unit tests: I thought I could ship whatever and the tests would catch it. That was incorrect; while unit tests are a guardrail, I still had to properly test my code.
Similarly, people need to learn how to use AI and make sure they don't introduce bugs or slop into the codebase.
But this brings up a larger point for me: most people are grieving the loss of what their jobs used to be, and denial is a core phase of grief.
I went through it too. I tried to find ways to say AI was silly or wouldn't work. But now it's inescapably here and we need to work in the new paradigm. There are plenty of valid criticisms, but it's already very good, it's getting better, and you have to learn how to use it as best you can.
There's a lot of slop, and it will almost certainly grow quicker than non-slop, but there is a big opportunity for those of us who still work at the bottom to make the pillars more solid, and take on re-engineering projects that would be infeasible otherwise.
Case in point: Before Claude a port of my compiler to a new architecture might take anywhere from months to a year. After AI, now that there are multiple backends as examples and an even larger--massive even--test suite, this can take days or less. It took Claude about 3 days to come up with an arm64 backend for Virgil (around April time frame). Then in August it took just 3 hours to build a complete RiscV backend. Both of them pass the entire test suite. And they shook out other bugs elsewhere. Overall, more targets, and more robust.
Things can have rocket boosters, you just gotta know where to place them.
LMAO. They can't lose something they never had in the first place.
This article has other pearls like this:
> Reddit and Bluesky speculated that even MSFT engineers had no real handle on the complexity of this outage.
Like, with all due respect, what the fuck do they know?
Also this:
>It’s the software that controls your flights, your banking, your petrol pumps, your TV, your payroll.
Dude, how are you gonna sneak TV software into the same sentence as flights and banking? :D
This is not a serious article, and I don't recommend reading it. But to finish on a positive note, since I've already (regretfully) read it, here's a sentence I like:
> This is the real threat: a scenario where nothing in the system is done with intent, and nobody in the system has control. We generate complexity faster than we can tame it.
I would just add that this was the case long before AI. And also, just as AI generates complexity at such a great pace, it can just as easily help you fix it and fight that complexity.
1. go fast and break things became the norm not the rare exception, mostly due to under-regulation when it comes to responsibility and other much more complex side effects of what you can describe as the US forgoing a fair free marked in favor of out of control monopoly markets they can use as soft power against other countries
2. software being really hard and most tries to fix that being stuck dead before maturing due to them being unfamiliar and often complex to get started with. There being a huge problem with opposite bad ends of "misguided simplicity" and "programmers loving 'clever' & overengineered" didn't help exactly either.
> It wasn’t always like this.
no it was, yes the 2010 felt like a high time. A lot of 90th nonsense has finally been left behind (but would still lead to security issues years later, e.g. log4shell in 2021); Tools for analysis had gotten better; Compute allowed expansive test suites and "go fast and break things" hadn't yet been spread to widely to critical infrastructure (at least kinda, _a lot of the issues with Azure today root back to decisions done in the 2010th_!!).
Most importantly while having well educated highly skilled Devs is needed, _it isn't sufficient_. They still are human. Blaming the huge flood of CVEs on AI generated code is simple, but wrong. Many of the CVE in recent years are human caused, some older then usable AI code gen and at least in case of the Linux kernel much of the code of the past had been written and reviewed by pretty professional, skilled, well educated developers and still AI found way way to many CVEs in it.
So yeah IMHO there is a long standing fundamental issue with how to handle software production, starting with the very first step of requirements engineering (which often isn't done, or at least not properly) and ending with how we fundamentally compose software components.
And the problem I see with AI it makes it worse. We already where approaching a breaking point with software dev and I hoped that this would create pressure to finally find a working way to reinvent it from scratch. But now AI allows us to apply the same failing approaches even more large scale, faster, cheaper and sure with more bugs, but also with AI making many bug fixes far faster and cheaper (if correctly used, it often isn't).
I highly doubt this is going to happen, guys. The world has changed and software is never going to be the same. The same battle against complexity and hard things will continue to happen, but the front has moved.
Open source as we know it is probably dead. Tech companies that would create SaaS for X and get billion dollar valuations are also likely dead. DevOps is dead. Other stuff will take all these things places in time but it's going to take probably a decade. Get used to it and let's get on with it.
Which means all critical software has bugs!
There's also more of it! Even more once you finish reading this sentence. If security incidents scale sublinearly vs the growth of critical software, we should be proud.
Crappy work is not universal either. There are engineers out there, guiding agents diligently, doing a good job. Never letting an agent go wild unattended, writing the same code they'd have written themselves, but with more tests and more better assurances and more docs and better infra and and and.
The problem that leads to blog posts like this is that good, diligent work, like critical software that does its job, are invisible.
Because no one notices a building that's not on fire. They just walk right by.
Sure the MS thing might be vibe coded horri, but it's not like we have not seen outages like that in the past.
If anything AI is bringing all of the bad configs and legacy issues to the fore front, because "it" has the time to look for them.
But look, the reality is we copy-pasted just as much slop we didn't really understand to get over the finish line back then as kids to today.
Deadlines were missed. There were bugs. Things broke. That's just life.
Now, do I wish we had TDD for everything? Yes. Do I wish there wasn't quite as much emphasis on AI? Also yes.
But there's a literal mountain of slop code that was written over the last 40+ years. Without AI, it sort of just sits there... people are afraid to touch it.
With AI, and I mean me using AI, I'm comfortable diving into a massive legacy code base and refactoring pretty much anything at will. And it's pretty cool when you stop looking at code as "I don't want to be the last one to touch it, because then I'll get blamed if it breaks."
And instead take the approach, "OK, without proper maintenance, this dies. How can I give it proper maintenance?" And look, if something breaks along the way, "Meh." I can fix it. Those old code bases were often just as buggy and duct-tapped as anything AI created... worse... years upon years of multiple developers doing patches. Convoluted AF.
I still like to think I have good discipline. I have a repo that has all my coding standards, and that's a good starting point for most of my projects. I weave in a bunch of rules as I go, things I learn, and that repo becomes the guide for the AI to do work on the project I'm working on. Then... when I'm not the one doing the work, it'll still be there to explain to the next person (or rather AI being managed by a person) why I did things, and what the goals were, and how shit works.
Most things that break... it's just because it wasn't documented in the first place. I find a lot of that... maybe they had discipline, and maybe the legacy systems that had the documentation were lost to time. Sure. Let's give them the benefit of the doubt. But... I'm guessing they did quite a few patches with just someone calling, "Hey Swanson, make the button blue, and make it say 2+2=7 to impress this client!" Right, so there's still a lot of that today too.
I am going to claim, AI is at its worst. Software companies are doing mass layoffs, they are decreasing the quality of software and reeking the profits, making sure only upper management (or worse, shareholders) are benefiting from software becoming so much cheaper and shittier.
And software workers are not loosing our jobs equally. Those that don‘t want to use AI are being fired much faster than those that do. And those that remain are de-skilling them selves, fast, letting AI do all the work for them.
I want to blame AI, and I want to blame capitalism and the profit motive for making sure that AI would inevitably cause as much damage as it possible could.
I could stop writing at this point because that's all you need to know to understand why "software is breaking down".
Anyway, the problem is just that there were never enough skilled developers while the demand for software was ever growing. That's the reason why the software industry has always been crowded with unskilled amateurs doing grunt work. Amateurs were tolerated simply because of the lack of highly skilled developers: one good developer can easily replace 5 untalented code monkeys, but it's better to have 5 untalented code monkeys than 0 good developers so that at least something gets done. Programming languages like PHP or JavaScript lowered the bar, attracting even more, even less talented people. It really started going downhill during the dotcom bubble, well before cloud computing was a thing, and it never stopped going downhill.
In such an environment, if you want quality results, the amateur work requires supervision and strict quality gateways so that good code can be distilled from the bad. That is, code monkeys must be managed and supervised by more disciplined, more skilled, more talented developers who are actually capable of understanding and managing the complexity of software.
These days, however, everybody is being dumbed down by AI. Everybody even seems to enjoy being dumbed down by AI, even those who actually are (or rather, used to be) good developers. Instead of doing proper code reviews by a professional, an AI is consulted to do the review. Instead of planning a software architecture, people are vibe-coding. Instead of doing a simple change that actually works, an AI is asked to generate spaghetti code (and you don't recognize the spaghetti because you don't understand your architecture anymore). Good practices that served as safety net against low software quality are ignored because "AI doesn't need them".
Of course software is breaking down, now much faster than ever before! It is because we (as an industry) are moving at neck-breaking speed and have stopped thinking, even those few of us whose job is to keep the quality at least at an acceptable level.
Spaghetti Claude is killing quality faster than most can realize, and it shows already. Your amateur code monkey crowd has been replaced by even less talented, but much faster AI with a knack for convincing language. Your code monkey crowd is still there, but it is now instructing the AI to produce results the monkeys don't understand and push to production. Quality gateways have been dumbed down or simply removed to keep up with the speed.
It's a bad development that needs to stop.
Leftpad, mentioned in TFA, is a case in point: not because of leftpad itself but because of all those defending leftpadism.
"People can't write a proper leftpad".
Yeah, well... If a dev cannot write a proper leftpad, then it should never have been a dev in the first place.
Someone who cannot write leftpad is not qualified to use leftpad as a dependency (and, no, leftpad is neither the Linux kernel nor a cryptographic libray: it's fucking leftpad, which is the entire point).
We've had "devs" whose only qualification was writing glorified "punch the monkey" ads in JavaScript: the JavaScript surveillance/ad complex. Devs who only ever knew the mono-threaded JavaScript ecosystem and suffered from heavy leftpadism.
And those very same surveillance/ad complex bros are now generating countless crap with LLMs, in languages they don't understand.
They're just mediocre.
That's what happened: the veneration of mediocrity.
Because of this, battle-scars from the rough edges of software are seen as part of software-competence, not as a failure of industry. Novices get met with 'you're holding it wrong', but at the same time veterans lose the plot and spend most their time doing technical onanism, or worse, fighting meaningless complexity.
Until the stack exceeds any individuals understanding, and the cracks start to show. Leaky abstraction upon leaky abstraction upon layers and layers of ducttape, under the guise of HA, distributed system mantras, robustness, you name it.
Every part needing babysitting because every part is half the solution to half of your problem, but admitting that is seen as a skill issue.
Yes my DB dies because it caches stuff and (under obscure circumstances) never frees it: 'thats just MySql, deal with it', yes of course my frontend framework needs to be kept up to date every week because everyone is evalling strings all the time, 'don't you sanitize your input?', multiplied by every layer of the stack.
Don't you let your static analysis automatically merge security PRs so you can stay on top of this giant heap of string evals? No? That's best practice man!
um what? who? where?