Hacker News Reader

Comment by prerok | Hacker News Reader

Are you referring to "move fast and break things"?

If so, it's definitely newish for those of us who started programming twenty+ years ago. Back then waterfall was the way to develop applications. Once it was burned to CDs or DVDs, that was it. We could not afford shipping broken things.

Ever since internet became ubiquitous we moved to on-the-fly patching for installed software and fast deployment cycles on cloud.

That said, the lack of forethought in design and backward compatibility is nowadays definitely on the rise and occurs likely on an order of magnitude more often.

View on HN

No, I'm referring to the "we're a massive enterprise and we don't make money by actually figuring out issues, so if service XYZ has troubles after 30 hours of uptime, then we'll restart it every 20 hours and problem solved" phenomenon.

Or the phenomenon where I piss blood explaining why having a p99 that is 30x worse than our p95 is maybe possibly a problem that should be looked at.

Or the phenomenon where concurrency exists, and so issues are no longer reliably reproducible, meaning everyone just throws their hands up and tries to ignore and downplay them as much as humanly possible. That is until a dickhead like me comes around, and does something like a scripted 300 restart cycle test overnight until a clustering resiliency defect finally reproduces, and i can capture enough debugging data that would never be possible on a live environment. All the while the product vendor is twiddling their thumbs, waiting for us to provide said data on a silver platter, because for some reason this completely stock issue doesn't reproduce on their end in a pretty much identical environment, or so they say.

Actually, I think we are in agreement, just saying things slightly differently.

I feel your pain and have had the same sort of issues. I once spent 3 months on a hard to reproduce issue to prove it was the closed source software from a vendor that was at fault.

Bugs have definitely been present since the start.

However, since we moved away from waterfall (not that I really miss it), systems are not that carefully designed anymore. Even if at the start the system is well designed, careless additions start creeping in and before you know it there are bottlenecks and race conditions everywhere. I think this was caused by the "move fast and break things" and the reduction of QA teams.

In other words, if it works in my small and non-representative environment, ship it. If it breaks, someone will tell us and we will fix it. I don't like this either, but yeah, it's been like this for a while now.

Latest development is IMO moving even more in this same direction, so it's getting worse rather than getting better :(

Reply on news.ycombinator.com