Hacker News Reader

Comment by benhoyt | Hacker News Reader

What do you mean by "do devops correctly" to avoid SSH on boxes? (I'm a developer, not devops.)

"Devops" has bazillions of meanings, but avoiding (human) ssh to production boxes is a generally sound principle these days because our infrastructures are becoming harder to understand by poking at boxes one or two at a time now even for forensic analysis.

So logging in to a server to check a logfile (assuming i dont or cant do centralized logging) is considered anti-devops ?

Edit: Sorry responded to wrong parent, sigh.

It's just a matter of scale. If you are at the scale of netflix, vm are probably too complex black boxes, and logs output somewhere else anyway. Plus the problem may involve the interraction of several vm, or the network, or other composants together.

At this scale, you basically have to have centralized logging. When you have thousands of parallel instances of a single application, searching logs box-by-box just isn't practical.

Consider also that if you're elastically scaling EC2 instances and you need logs off an instance that's since been terminated, too late! That disk is gone. So again, you need a central log service.

Parsing logs isn't the only way to troubleshoot.

It's not even the best way, just sometimes it's the only way.

Immutable infrastructure [1] is preferred. If SSH is detected, you must assume something on the server changed, and has deviated from the baseline.

[1] https://www.oreilly.com/ideas/an-introduction-to-immutable-i...

If deployment is automated and "clean", images get baked into machines and they just start. For instance, we use Ansible against the machine itself on boot, so we don't really need ssh access to it: everything is automated (but we keep it open to troubleshoot anything that may happen)

Reply on news.ycombinator.com