Ancient Linux servers: The blighted slum houses of the Internet

As for trouble you've had with AWS, I'd encourage you to take a second look at S3 specifically. It's been pretty bulletproof for a long time now.

To be more specific:

An architect "lives" in the development staff: they're aware of the internals of the application. Of the system, the architect specifies what must be true of the stack the application runs on--and this can be quite detailed--and might participate in bringing that stack up. But in the system's day-to-day life, the architect need only be aware of APIs for interacting with that stack (which they presumably specified needed to exist.)

An admin, meanwhile, "lives" in the ops staff: they're aware of the internals of the stack. Of the application, they are only given a set of APIs it expects to be made available to it; it's otherwise a black box.

The architect writes scripts, which run alongside the application, to maintain the application's presence on the system. The admin writes scripts, which run directly on the system, to maintain the system within the reference range specified by the architect.

Picture, for example, a de-novo implementation of Search at Google. The Search application has an architect, who specifies what sort of servers Search needs to do its job. The Search architect sends this information off to ops. Ops designates someone "the Search sysadmin", and that person constructs a server farm. They script the farm to correctly cluster tasks, maintain its own health, etc. and train operators in the management of that farm.

In my mind, it's really clear if you think of servers as running something like Docker containers to contain the application: everything outside the container is the sysadmin's job. Everything inside the container is the architect's job.

I guess I mean more by "automation" than you do, though. Paying some other company to do things like software and package assessment for you, so you just have to consume their customized software distribution and can let the system update whenever it likes, is "automation." Buying standard hardware that you can find pre-tuned configurations for is "automation."

Effectively, anything that moves the company away from maintaining a "system", and closer to being just a tiny core of people who run one tiny thing on top of a large set of frozen, well-known components is "automation."

That said, these factors really complicate security advice on patch management. If customers could be trusted to lock things down and keep an eye on them, that would be a much better world. And I'm sure a lot of admins out there are more than capable, but I worry about the Dunning Krueger effect catching some admins off guard.

But ultimately, this is just a battle of emphasis more than disagreement. The answer isn't "everyone should always patch everything," it just depends on a lot of factors.

> Everything you said is very general.

Ok, here's specifically how this approach fails in the real world:

A guy is ignoring vulnerabilities that don't seem to apply to his configuration. So there's a kernel flaw that allows privilege escalation. He thought it was no big deal because he doesn't allow a guest login. Then there's a flaw that allows remote users to trigger memory corruption, allowing remote guest access. No risk there, he thought, because guests have no privileges on the box.

You see how the attacker got in?

You might counter that that admin was just too "lazy" to line up all the vulnerabilities and see how they interact. But there were almost 200 vulnerabilities last year just in the kernel. Are you going to conduct the 19,000 security audits required to see how they interact? What about groups of three? What about vulnerabilities in other packages? This workload doesn't just go up linearly.

Also, this approach is weaker against unpublished vulnerabilities. If you're strapping together older software, especially deprecated stuff no one else is using, you're losing one perk of open source software, the many eyes shallow bugs bit. People not using your configuration means no one is going to discover vulnerabilities on it except for attackers. You may think, "good, that makes it harder for attackers," but that's security by obscurity, it doesn't actually hurt attackers very much. Vulnerabilities patched in newer versions of software give insight into vulnerabilities lingering in older versions, so often exploits can be crafted for previous versions far more easily than for newer.

This is why security professionals recommend defense in depth. You don't know which part of your platform is going to break and allow attackers to exploit vulnerabilities that you didn't think were relevant.

Also, for a lot of systems, eventually the admin will change. The guy or gal that follows you will be dependent on the system you set up, and may be less experienced or less capable. If they inherit a patch management system that basically entails, "Become a security expert in addition to your other duties," they are not going to strictly adhere.

You know your system and your situation, so I don't want to sound like your approach can never ever work. But if we're giving advice to the unwashed masses, I think we need more advice tailored for the people you dismiss as "too lazy to pay attention to security," because that describes basically everyone.

PS - When you describe other people's opinions as "lazy cop outs," it can kill discussions. You might watch out for that. HN isn't Reddit, people sometimes bury stuff that has good substance but poor tone.

At this stage, Windows XP users are like Japanese soldiers stranded on an island, growing old not knowing they lost the war. Except without the excuse of being on an island.

An alternate analogy would be something like: "I saw you forgot to install a lock on door, so I superglued it shut."