Comment by laingc | Hacker News Reader

Comment by laingc | Hacker News Reader

There’s a certain section of HN that has apparently decided that everything was better when C was considered a high level language, production was a single physical machine that you personally managed the cabling for, and 640Kb of memory was all anybody would ever need. This section of HN is either composed of god-tier ninja programmers, or of people looking back through rose-tinted specs and pining for “the good old days” - my guess is mostly the latter.

Maybe you run an amazing shop and everyone should be as good as you - I’ll give you the benefit of the doubt. But in my experience, every company I’ve seen that’s populated by people with this kind of attitude has been a raging dumpster fire, perpetually running a hair’s breadth from exploding violently.

The world writes, reads, and runs a lot more software than it used to, serving a lot more people in a lot more countries with a lot more variation in the circumstances of how it’s used. This adds up to a lot more complexity.

Docker is one very important tool that helps us to manage this complexity. Yes, it also introduces some difficulties that you didn’t have before, but it solves much more difficult problems pretty well. We use docker for almost everything, and it’s a lifesaver.

These people aren’t arguing that docker is a bad tool to manage complexity, they’re arguing that a lot of the time we build things that are unnecessarily complex. Most “scaling” issues are just performance bottlenecks that can be addressed with a profiler and some optimisation, yet the go-to answer nowadays seems to be to just throw more machines at the problem.

Probably because more machines is faster and cheaper than a profiler.

At least in the short term.

If a programmer is not able to write efficient code, they may not even know that their code is inefficient.

I regularly see programmers that come from a front end or "full stack" background try to solve data engineering problems, and end up with solutions that are 10x, 100x or sometimes 1000x slower than what they could be expected to.

No kidding.

It's not rare for me to catch Pull Requests with horrible performance holes. Like 'SELECT * FROM...' executed inside a loop where a single 'SELECT id FROM...' outside the loop could solve the problem.

> either god-tier ninja programmers, or rose-tinted specs

False dichotomy.

Removing some of the many, many layers of crud that hobbles our incredibly powerful machines does not take ninja-skills. It does however take the knowledge of what is possible, which is why it is often older developers who bring it up.

Tools that "manage" complexity very often beget more complexity that begets more tools that beget more complexity. Because each layer always turns out to be somewhat less good (and more leaky) at managing complexity than hoped for, and always turns out to add more complexity itself than initially anticipated.

This is not a new trend. We had "middleware abstraction layers" in the 90s. That kept me amused for longer than I care to admit.

So instead of trying to "manage" complexity, and usually failing, reduce complexity where you can. And as I wrote, it actually isn't that hard once you convince yourself that it's actually possible.

"You'll be amazed"

Say you're starting a new project, how would you keep the complexity down? It's a hard subject and I'll admit I'm prone to follow the interesting tech.

Use TDD and hexagonal architecture.

That is, start with pure model and build outward from there. Discipline yourself to build only what you really need using TDD: only write production code for a failing test case. Only write enough production code to just barely make the test pass. This will lead to silly code. Refactor this to sensible code when all the tests are green and you sense duplication.

Avoid starting with environmental things: infrastructure, frameworks, databases, etc.

Don't mock.

My graying computer organization professor always referred to the days of tech yore as “the bad old days.” I think that is something we should consider doing more. Things look simple in retrospect but in the midst of it, it is often complicated from day to day.

> We use docker for almost everything, and it’s a lifesaver.

While we're sharing anecdotes, we are using Docker for a part of our CI pipeline, and it's the only thing that constantly breaks for some reason.

My team as well has fixed a lot of flaky tests by reducing the number of places we use docker. It's the most finicky part of our infrastructure. I always wonder what it would be like if comments were tagged with the technologies in use by the commenter; would we find that the strongest proponents of docker are working in stacks that have a poorer isolation/deployment story? I remember trying to package Ruby apps and thinking "huh, this is a pain." If that was the world I was in, I can see why shipping an image of my OS is a lifesaver.

Yes, that's a good point. I have two Rails sites (side projects) and I've never worked with something so finicky. Something always breaks, and after every OS update I habe troubles getting my dev environment back up. I can see the appeal for Docker there.

But with all my other stuff this is really not an issue. I have a few small services written in Python, and some old stuff in PHP, and that stuff is so easy to run and deploy on pretty much any OS, and adding Docker would just add pointless complexity.

Reply on news.ycombinator.com