PagerDuty hacked. And finally comes clean 21 days later | Hacker News Reader

PagerDuty hacked. And finally comes clean 21 days later

theregister.co.uk

112 pointsGarbage17 comments

Has anyone written a checklist for CEOs/CTOs of stuff to do/avoid when such incidents are discovered?

I would love this from someone with the know

In my opinion: the Pager Duty team did okay. Disclosing sooner would have been cool, but then again law enforcement might have had the gun to their heads on disclosing anything.

They explained their bcrypt implementation awkwardly, though. This is what their code did: https://github.com/plataformatec/devise/blob/1a0192201b317d3...

I would not have advised them to go the pepper route, but it's probably a moot point now.

In their favor: When asked, they gave specific answers up to the point that they were allowed to by law enforcement. More companies should follow their example here.

> Based on the investigation, the attacker bypassed multiple layers of authentication and gained unauthorised access to an administrative panel provided by one of our infrastructure providers.

Sounds like a VPS provider screwed up? If that's the case, would love to know which one it was. :/

They have a couple of subdomains hosted with AWS, in the us-west region looks like. Not really definitive, but AWS is common infrastructure for a lot of startups these days, and it wouldn't be the first time that someone managed to find their way into an AWS admin panel.

Linode seems like the more likely culprit here.

Given Linode's past history, I'd regrettably have to agree. I didn't see evidence of Linode infrastructure when I posted my earlier comment.

It's frustrating that PagerDuty can't / won't share that information. I can't see how that's helping law enforcement's investigation, and it's potentially putting a lot of other people at risk depending on the nature and severity of the compromise.

Or maybe they had a weak password for their administrative account@VPS?

It does not immediately mean that. If it were AWS, there are ten thousand things PagerDuty could have done wrong: lost an access key, made an IAM role too permissive and lost access to the instance, turned off key-based ssh, made the ports open to the public, hard coded credentials, simply used a weak password for their console, not enabled TFA, lost the root credentials somewhere, allowed a privilege escalation by giving one key the ability to create more keys and attach permissions. The list goes on.

Plug: I'm working on an open-source and hosted project called CloudSploit that can scan for these kinds of things. Check it out at https://cloudsploit.com

Recent discussion on HN: http://news.ycombinator.com/item?id=9977399

This is way, way too prevalent -- people thinking that by doing more things to the password you are being more secure. The bcrypt algo already salts, there's no need to prepend extra stuff. And if you really had 80 chars of preliminary table condiments and your bcrypt implementation truncated at 72 -- well that's just textbook.

Just PR content, too little, too late.

now wait and watch if someone will dump this on pastebin or some torrent.

compare it with what slack posted under similar scenario

http://slackhq.com/post/114696167740/march-2015-security-inc...

It was less than a week ago that someone here proclaimed loudly that 'security is just as bad as premature optimization'. I wonder how the pagerduty folks feel right now, but I'll bet that 'premature optimization' is not the first thought that comes to mind.

A good start-up fires on all cylinders and strikes a balance when it comes to features, user friendliness, performance and security. Ignore any one of those and you'll lose the game eventually.

Let's hope they pull together and up their game then manage to survive, I'd trust them much better after failing like this than before assuming they learned their lesson.

>It was less than a week ago that someone here proclaimed loudly that 'security is just as bad as premature optimization'.

Although I work in infosec, I could see how this case could be made if your company/service does not handle or store any customer or financial data at all, and does not have any particularly sensitive information. Otherwise, it's a pretty ridiculous statement.

That definitely wasn't the context. I won't link to the comment here because I think it is an embarrassment to the maker but it's easy enough to find if you want it badly.

Yeah, jacques is definitely pulling my comment far out of context. I was commenting on his suggestion that everyone start serving their own JavaScript files, and writing custom code to check their sanctity.

I can't see the parallel here, or how that practice would have helped PagerDuty.

* cue PagerDuty "sad trombone" alert sound

Reply on news.ycombinator.com