Are we at the point where the people who maintain the infrastructure are now completely different than the ones who built it, and are struggling to keep it running because they don’t understand it as well?
Are we at the point where the people who maintain the infrastructure are now completely different than the ones who built it, and are struggling to keep it running because they don’t understand it as well?
Look at some of the code that was open sourced by Yahoo years ago. Other large engineering organizations today have technology which far exceeds the complexity of what was made public by Yahoo a decade ago. Unfortunately Yahoo is pretty much the only example of a large company open sourcing big portions of their platform code.
Twitter used to go down so frequently the "fail whale" was a whole cultural thing.
Big AWS outages are rare, but hardly new.
https://www.theregister.com/2017/03/01/aws_s3_outage/
https://arstechnica.com/information-technology/2012/10/amazo...
So I do think tech debt is accurate, in that the average human deludes themselves into thinking they will pay it off at some point.
I'm not sure tech debt can ever be fully paid down. You can pay off a bit, and you can stop the debt from accumulating further, but the only way to realistically unburden yourself of a really horrible, big pile of tech debt is to just rebuild the thing from scratch. Or don't, and just keep making money while you wait for the business to fail. (From an investor's perspective, this is the equivalent of riding the car into the ground.)
Every service you mentioned (maybe sans Gmail, the only additions I remember in the last twelve years are a new UI and Hangouts) has bolted on so many features over the last years: AWS was virtual machines + SDNs in the beginning as Amazon only intended to sell spare capacity on their own servers, now it's a global one-stop-shop for everything that can be done on the Internet. Facebook was a social media feed, now it's event coordination, groups, chat, image and media hosting at global scale.
And apparently, no one at these organizations ever thought about re-working their infrastructure with "lessons learned over the last decade" in mind. Every new feature was simply bolted on, on top of an infrastructure that was hardly even envisioned to ever become the scale they are today. And that sort of refactoring costs serious amounts of money and developer time, not to mention that it doesn't make sense to develop new features on a code base that's going to be shut down in a year, so management doesn't approve it out of a fear they will be "out-featured" by a competitor and cannot react (=copy, like Instagram's Stories that were a clear rip-off from Snapchat) in time or that their own PKI/OKR goals and with it their bonus payments won't get hit.
That mindset/scale issue is also why IBM mainframes are still so common, why travel PIRs seem to be stuck in formats over half a century old or why "put CSV files on an FTP server" is the standard on bank transfers... big corporate/government clients pay a shitload of money for virtualized mainframes on new hardware that still can run the 70s-era code and even more money for people able to speak COBOL, because that is still cheaper than the alternative - reworking everything from scratch on a modern foundation, testing data integrity and edge cases, revise interfaces to hundreds or thousands of clients. Hell, even Internet standards have the same problem... we are still using protocols like BGP that have been around since before I was born, and tacked on security only a few years ago after a couple of fat-finger incidents.
Modernization in such entities only tends to happen when laws or regulatory frameworks change, and then it can become a real shitshow for those at the lowest rungs of the IT ladder that have to implement them - simply take same-sex marriages and try to shoehorn them into a database that was labeled for "husband and wife", or trans/inter people with gender data represented by a boolean field.
Of course you need to make sure the network is actually decentralized (see Solana). Or maybe you can just rely on Amazon and GCP and Azure, because surely they won’t all fail at the same time.
You usually just don’t need 100% uptime. “Sorry were closed, come back later” is fine.