There's always hope for things to reverse course and get better. Ruby on Rails was a moment like this in web development history. Rails came at a time when things were unnecessarily complex and introduced a much simpler way, which people accepted. Rails' influence extended far beyond Rubyists and introduced design patterns that became part of web frameworks in every language.
Rails had a bigger impact than anyone cwould have anticipated from a quirky new framework in an unknown academic language at that stage in the industry's maturation. Computer scientists had expected that by 2000 every task in software development would be catalogued and assigned a standard labor estimate like in auto repair. It was highly improbable that an entirely new paradigm would take hold after that. I think part of this was because of the technical ingenuity of the solution, part of it was because of the charm and personality of the Ruby community, and part of it was synchronicity with other movements like Web 2.0 and Agile.
It takes leaders with strong personalities who can attract people and present new ideas in a way that people will accept. They also have to have insight into what people actually need in their time and willing to compromise.
Throughout my entire career everyone has been talking about reducing complexity and increasing speed of development. But what has happened? The opposite.
At a place recently, having a bunch of microservices all connected over the network ("distributed monolith", mostly written in Java), for example, you now have to worry about turning your internal data structures into JSON, worry about deploying all these components separately, getting the development environment for each one set up on your computer, etc. You also have to worry about idempotence and retrying requests which fail due to network connections etc. Vs. just calling one Java function from another.
Who makes these decisions? The architects and senior engineers. What are their objectives, if we're truthful? Them getting payrises and not getting fired. What are the the things that might cause them to not get payrises or get fired? Juniors replacing them and driving salaries down, or new seniors coming onboard and them no longer being the "stars". Having an incredibly complex architecture, which only they know, benefits them. So if they benefit from complex architectures, and they also get to make the decisions about which architectures get used, we shouldn't be surprised that complex architectures get chosen.
People talk about simple architectures, but actually it was never really about creating simple architectures. The incentives are not set up in such a way that simple architectures would result.