Hacker News Reader

Thanks for that perspective. Do they ever find time to do side-projects? It sounds like these people work 24/7.

I'm a devops engineer (I guess we're called SREs now too) at Localytics. We're not Google-scale, but our stuff isn't small, and a big focus of what we're doing is making sure we don't have to work 24/7. Automation, automation, automation. Handle failure states automatically and only inform humans after the system has fixed itself or if the system can't fix itself.

It's pretty fun.

How do you control the quality of that automation code? Doesn't that add an overhead to your operations?

Good testing. I got hired as a Test Engineer in SRE at Google to write automation code tests. Good coverage from unit tests, good integration tests, and good system tests (many of them using fake systems with scripted failure modes). Then finally, a fair amount of testing on "real" machines as new code gets integrated.

Then make sure you have great monitoring because it will still fail at runtime and you're the poor sap on call who has to deal with it.

Absolutely. They're busy but work on side projects. Depends on the person.

Do they usually pursue side projects that help them get better at their troubleshooting/automation skills or do they tend to keep them complementary to avoid burn-out?

yes

Reply on news.ycombinator.com