Or is it some kinds of fallacy to believe that simplicity has a rather obvious form?
It's more like "simple" isn't "simple enough", isn't it? The latter of which one can get to faster and from his own POV alone while true "simplicity" takes time and consideration.
Consider how much more efficient we should be with allegedly-better processes and allegedly much better tools.
Look at some of those and estimate the team size and time to complete it now. Consider whether you could even deliver the same thing (say, native applications on two or three platforms) with double the team size and thrice the time, or whether you’d have to compromise and deliver something worse to make that happen.
There’s your answer.
[edit] though actually I think part of this is also because we’re on-average worse at managing companies overall, too.
Today I work in a team that supports a wide array of operating systems, device form factors, resolutions and aspect ratios, input methods; we maintain the backend and it's up to us to make it secure, performant and fast for users from anywhere in the world; and of course API access. Customer data doesn't fit on one disk and their processing can't be done with one machine, and they have a globally distributed workforce. We are slower because we do much more.
Games yesterday: worked on certain HW, sound probably required you to jump through hoops to get working, and networking maybe worked on a LAN. If things crashed it was fine because that’s the only thing you ran anyway. Cutting edge graphics for the time but at that time showing an image or playing a video was also an achievement.
Games today: supports HW from at least 2 GPU vendors, an insane amount of resolutions and typically basically the same code for PC, PlayStation, XBOX and maybe even Linux and Mac. Sounds and graphics just works. Networking lets you play with people across the continent over the public internet. Oh and the time and effort invested in graphics quality is insane.
Teams yesterday: highly motivated engineers who were extremely talented and new every part of the machine
Teams today: a mix of highly motivated engineers who are extremely talented with many more who just want their paycheck and to go home of very varying quality of talent. Same expectations of excellence. Machines have gotten very complex that cutting through abstractions is challenging
Infrastructure yesterday: sell a database and the business is responsible for maintaining it and having it run well with onsite domain experts to help
Infrastructure today: sell almost anything so that you need almost no understanding and expertise of the software you’re utilizing.
These two are not mutually exclusive.
You missed a key part of that sentence. Also, there’s a huge difference between two engineers when one is super motivated and engaged by the work vs someone phoning it in.
We are slower because we do much more.
But hardly anybody would buy it or want it.
Today, a team of 40 struggles to release a workable game on one system that has to be patched on day zero because of all the bugs. And porting it to even one other system (with the same CPU architecture)? Forgetaboutit.
You can argue that the game companies are focusing on hyper realism at the cost of fun games, but the games we’re playing today are orders of magnitude more complex than in the Atari days. Those Atari games built on today’s technology would take comparatively little time and could be as easily ported. That likely also explains why the game industry is focused on limit pushing AAA titles. It’s a defensible market position of technical advantage whereas competing on “most fun” puts investment dollars at risk because you can be undercut by an underdog more easily. So the big companies just focus on the places where it’s harder for upstarts to compete and why almost all games look and feel very similar.
Once they bring Basel, Go, Tensorflow and C++ into projects and start refactoring and perfecting theseNamesThatJustHaveToBeCorrectAtTheLevelOfExcellenceThatCanGetAPlusTwoOfXooglerPrinciple then their complexity and costs get to the level of a software engineering project at Google. Without having the quality of engineering workforce and infrastructure that Googlers enjoy.
Not a change to the better in the software sector management, in my opinion.
Then: Support a single platform, Windows or Mac
Now: Support Windows, Mac, and Linux, ARM or x86, as well as mobile.
Then: Support a single market
Now: Support a global market with region-specific localization
Then: Write software for wealthy and highly technical early adopters
Now: Write software that anyone in the world could possibly consume
Then: No accessibility
Now: Accessibility requirements for software being codified into law
Then: Zero or low consumer expectations, win by creating something new (blue ocean, lotus123 versus literal paper)
Now: Nearly everything has been done, win by differentiating on comparative quality in a field of options (red ocean, figma vs keynote)
1. A need for standards
2. The industry being too immature for standards to exist and those that are created rarely stand up to the test of time
Look at standards from the late 80s / early 90s, most of it was not prepared for what software has NEEDED to change to.
Look at most programming languages, designed to only run on one machine, when a vast amount of software is designed with high-availability in mind.
Look at most databases, designed to have only one writer which is a huge bottle neck.
Now we need software to coordinate all this software that was designed to run by itself...
Finally, in the pursuit of perfection, we test new ideas in the open, the latest UI framework, a new database, a new paradigm... We are doing the scientific method on NPM and your organization is a part of it :)
Some of it is incompetence.
Occasionally there are claims that it's deliberate attempts to make the author look more valuable though I haven't seen that in practice. Hopefully it's a myth.
Some of it is indifference. You're sufficiently beaten down by the environment that you no longer care about the product.
Also the underlying problems are sometimes very complicated in which case there may be no way to make the solution simple.
But all of your points are still valid and important parts of the problem.
That’s entirely due to “the cloud” and k8s and friends.
But I think the industry is waking up to this; there's just a lot of sunk cost. Kubernetes makes a ton of sense, for some organizations, but if you're not hosting your own machines in a data center somewhere, its probably not for your organization and you'll always be better off just using the highest abstraction level non-k8s primitives your cloud provider offers.
React: Is in a different league. React solves legit problems. UI is hard. React doesn't create zero problems of its own, and some of those problems are of the "we thought redux was the best way to think about this, we were wrong, but now there's a billion lines of redux out there" variety, which sucks because it was self-inflicted. Also, you've got the Vercel folks entirely convinced they're God's Gift to Humanity steering the ship in directions no one really wants. But the core of React is really good: fn(state)->ui.
Whatever it’s evolved into now is a different monster.
That gut feeling only comes through pain, aka experience, botching a refactoring you thought was brilliant to simplify code, and realise there were reasons for it to be overly complex. Experience working with overengineered code that takes a while to realise it's actually much simple than what the code is telling you, and can be thoroughly simplified.
Over many cycles of those experiences I believe one can start developing a sense of form for what's good simplicity.
Another point is it comes from the experience of reading vast amounts of code, from many different skill levels/sets.
Simplicity takes time and consideration from experience, I don't think you can jump the step of building experience to be proficient at it.
In the '10s, everyone was talking about "webscale". VCs noticed that we could write software for hundreds of thousands of occasional users instead of a core group of a few thousand users.
We thought we were making UIs to help people do stuff, but we were really making little feeler apps for vertical integration for corporate interests.
Stuff like big-data and kubernetes looks overcomplicated if you believe the sales pitch. The "tech set" is negotiating for control over business.
- we're not shipping a final product; this needs to change in the future very quickly
- we don't want to give you two years to build a feature. Build the start of a full product now and iterate with no break
- we would like a pixel-perfect custom UI instead of CLI or whatever Windows drag and drop UI builder would give us
- we want to run on multiple OSes
- we want to run on tablets and phones as well as desktops
- we want to have translation into every language and alphabet, rather than staying in ASCII
- we want cloud backups, and configured to the user-privacy compatible regions of the globe
Don't you think?
Software engineering on the other hand exists in a parallel space that is perfect and idealized. It's engineering that never has to leave the textbook ideal models. Mother nature has no claim in this space. This doesn't mean it is necessarily easier however. It means that all the effort typically put into just making something not burn up can now be put into complexity. So you end up with systems that are insanely complex. Engineering minds unchained by the constraints of reality, you just get complexity built on complexity serviced by complexity.
You get what you reward. Our industry rewards more keywords in your CV. Therefore, people try to get as many keywords as they can.
6 or 8 years ago, I was a second level manager looking to find the next managers in my group. There was an engineer who was great at mentoring and was exceptionally practical. What do I mean practical? I repeatedly saw her going to her PM and saying things like, "If we can skip this one feature, we can get the other dozen done in a week instead of a month and there will be less than half the code to maintain." The PM was more than happy to do this because he had a lot more features in the pipeline that he wanted to get to, and giving up that one meant getting a bunch more.
That engineer initially turned down becoming a manager (a couple of years later, she became a great manager!). Her reason was that she felt she was the least capable engineer on the team, and she wouldn't know how to coach the better engineers. This confused me, so I asked why she felt she was the worst. She literally said, "[name redacted] can stay late and write a 1,000 lines of code in an evening. I haven't written a thousand lines of code in the last quarter." I had to explain that it's entirely possible that that person is a better programmer, but not a better engineer - they are not the same thing. And, that prolific programmer hadn't shipped a feature in months.
Features, at least good ones, are an asset. Lines of code are a liability. You need the features to be valuable enough to pay for the ongoing cost of the lines of code. This engineer understood this intuitively, without realizing it consciously. The prolific programmer did not get it at all.
So, even as I was trying to build a culture of better engineering (i.e., finding better trade-offs, including limiting complexity), these broad ideas about complexity==good were in the way, deep inside people's heads. Being able to foresee and head-off complexity is an incredibly valuable skill. Sometimes the complexity is worth it, but so often it's not.
if someone isn't knowledgeable about the problem they're not going to be able to select the most simple solution.
if someone isn't knowledgeable about the available solutions there's a higher chance they'll implement an unnecessarily complex solution.
now multiply this over time and you get a code base that is really good for everyone's job security.
multiply this by millions of code bases and you get a really good tech economy.
That: the simplest solutions look obvious once you see them, but they’re rarely, if ever, obvious from the outset. Simply put, the simplest solutions are rarely the most obvious. You have too actively look for them, and that takes time and effort.
One corollary is that people who take the time to design the simplest solutions that end up making life easier for everyone else, tend to look much less productive than the tactical tornados that masquerade as a heroes. Worse, when the boss looks at their code it’s much simpler and much easier than everyone else. Clearly they took the easy route and chose the easiest things to work on.
And what do you think happens to low performers who chose the easiest work? They can leave, one way or another. But they can also give in and expedite more, more complex code.
Simplicity isn’t just hard. It’s a hard sell.
It takes more effort and time to produce simpler solutions, and generally speaking as an industry, we're way balacned toward speed over quality. We basically only care about "quality" as far as it has direct customer impact. It's relentless optimization for the short-term over the long-term, and it leads to complexity.
So you really can't master anything anymore, at least not like in the old days, because there's just so much more of it and you only have so many hours per week.
But, this is probably inescapable. We have the internet now. You generally just can't write a monolithic app. It's going to be two apps (client and server) or one app that has to think about both client and server.
And also Moore's Law went and died on us. So no more single-threaded... anything. Everything is threaded or parallel or etc etc now.
But today's "get it done now" outlook encourages complexity. That was always around, but ....
About 20 years ago or so, it was OK to get things done right and be a little late. Now seems this "get it in now, do worry about consequences", then "Well we have issues, create a request and will will get to it later". But later hardly ever comes, so the users are stuck.
So here we are, everything is complex, no time to think about what you are doing.
I guess the difference that (in Germany) math exercises cares more about reasoning and explaining while in CS / informatics solution is only the tested result. So yes, coder make the world more complicated ;)
That may explain the sentiment that the code is the documentation. There may be some html or pdf with "documentation" in the name, but if it's inaccurate, you're going to end up looking at the code anyway.
If that's your experience it makes writing documentation hard to justify. It's a poor approximation to reality which becomes steadily less true over time. So really what's the point.
We need something closer to live documentation which runs the examples against the codebase and fails CI when they stop working to have a chance. Some projects do that. Deriving docs from comments helps a little but comments are often wrong too.
People don't want simple programs today. They want bells and whistle and stuff that doesn't make them think. They want to be, in effect, 'mindless morons'.
In the old days, you had a very simple text program that did every single thing you wanted but you were obliged to remember the command-line options for it.
Today they don't want to have to remember anything, they want a button nicely labelled right in front of them to do the exact thing they want.
If you try to select body text, it starts scroling.
If you try to select bullet point text, it reloads the page.
You have to work pretty hard to break something simple that much.
Nearly 10 years ago, we had the left-pad incident[2]. As I post this, people are in a panic over a similar issue. This time it's called the polyfill incident. The details are different, but they don't really matter.
The root cause is the same:
1. People saw nice code
2. They didn't make backups of the nice code
3. Instead, they trusted someone else to keep serving it
This time, the trusted party served malware instead of deleting everything. Innovation!
A more serious (but still oversimplified) take: static linking and/or vendoring can mitigate this for a price:
* Vendoring is copying other people's source into your repo
* Static linking is the compiler doing the same thing with binary code
It's a trade-off:
* Pros: durability and longevity since everything's in one .exe / folder / zip
* Cons: lose flexibility and gain software license complications with the LGPL/ etc
It also isn't as trendy as "what you need to know" articles.
For educational yet actually funny takes, I suggest reading these:
* The Grug-Brained Developer[1], a light-hearted take about complexity being bad
* The famous ML / AI XKCD[3] about stirring big piles of numbers. What's changed since then?
* The size of the number pile
* How fast we (our GPUs) can stir it
* The amount of investor money having the tallest pile can earn
* NVidia's stock price
[2]: https://arstechnica.com/information-technology/2016/03/rage-...
“Simplicity” can also be defined any number of ways, but is most often defined as “Maximally pleasing simplicity” rather than “Fewest moving parts simplicity”, in part due to misestimating how total volume of moving parts is an exponential complexity modifier when one is not already familiar with the mechanisms. https://xkcd.com/2501/
A: "Job security ha ha ha"
(paraphrasing of an actual conversation I have had multiple times with coworkers)
“Hanlon’s Razor”
There definitely aren't enough people deliberately obfuscating to make it a generally visible cause of complexity; your quote here rings far more true to me.
For example, "refactoring" as a task very frequently just trades one set of things, often aesthetic things, for another -- it is rare to find any obvious best expression.