Hacker News Reader

Comment by yknx4 | Hacker News Reader

I'm baffled with the fact that they were toying with the production database in the production machine. That sounds incredibly dangerous

View on HN

Actually if you read the article this is not actually critical infrastructure.

All the data is duplicated in both DCs and the source of data still has a copy of the data.

I imagine is not a problem if it goes down for some moments. Developers may be unable to review traffic logs, that's it.

> I'm baffled with the fact that they were toying with the production database in the production machine. That sounds incredibly dangerous

As Taryn stated repeatedly in the post, they simply didn't have enough infrastructure to have development spare copies of this. They didn't even have enough infrastructure to take a backup.

Which is obviously nuts. If your developers are ever spending months of their time to work around infrastructure limitations, that means you need more hardware. You could build a new server to handle this job easily for much less than the cost of this person's time.

Unfortunately, the exchange rate for converting slivers of dev time into hardware is... Unfavorable. And up until two days ago[0], Stack Overflow was running on fumes - shedding staff, perks, equipment to keep the ship afloat. I know of folks who spent months scrambling to get new servers before the old ones died, and... Didn't quite make it.

Given the situation, investing time into making do probably seemed like a much better bet than hoping for a windfall with which to buy equipment.

[0]: https://news.ycombinator.com/item?id=23976881

Reply on news.ycombinator.com