Hacker News Reader

Comment by random42 | Hacker News Reader

I am a software developer, so I know "shit happens", but having the same configuration for database as testing environment, (same superuser name and password), which is not isolated from test environment, is pretty criminal even for a first time mistakeIMHO, especially for a product like github whom business, small and big trust with there business critical piece ("repository").

If I were running some critical code, I would have seriously reconsider github, or at-least ask for a detailed explanation on their engineering practices and fail-safe mechanisms.

View on HN

I can only hope it shocks some sense into kids that use GitHub for distribution rather than putting a tarball named $name-$version.tgz (or bz2 or xz or whatever). As much as I love GitHub, is has been the bane of my automated-build existence. I don't want to ever have to make a build script that guesses at a SHA1 (or punts by doing a clone at depth zero) again.

git ls-remote will help you to get the hash of HEAD. It's much cheaper than doing a zero-depth clone.

I think the fact that they have been able to recover all critical data soon and will even update the event table over the next few days clearly displays their competence.

Github isn't facebook and not seeing the last week's "activities" timeline for a few days isn't really that much of a problem.

> I think the fact that they have been able to recover all critical data soon and will even update the event table over the next few days clearly displays their competence.

Taking a few days to restore a table indicates they only had a text dump of it. The details of how they screwed up indicate they were new to the idea of data robustness, despite it being the core of their business.

Being a competent home user and being a competent data-robustness company involve slightly different levels of expertise.

Git itself handles robustness for what counts -- the source code and revision history. This outage did not affect repos as far as I have heard.

Backing up and restoring mysql is an orthogonal problem.

The fact that github can operate at all in a degraded mode very much indicates their competence. Many architectures can not operate without the entire db.

Plus I bet they'll shore up their db operations next. Github is very much a young company that will take a lick or two in their way to greatness. All the startups here will.

> This outage did not affect repos as far as I have heard.

Well, I'm just going by TFA, which stated:

"Newly created users and repositories are being restored, but pull request state changes and similar might be gone."

Repos were affected.

> The fact that github can operate at all in a degraded mode very much indicates their competence. Many architectures can not operate without the entire db.

This is a curious statement to make after you started off by recognizing that git handles itself. The only parts of the repositories that broke were the parts they tried to glue to their database.

It's nice of you to give them a pass. I'd say DBAs are pretty horrified. I suppose it's kind of like how the average person can watch CSI where they say, "I'll create a GUI interface in Visual Basic, see if I can track an IP address" and think it's just fine.

Can you share more details about how you usually work ?

Reply on news.ycombinator.com