Hacker News Reader

Comment by __jal | Hacker News Reader

There's also a weird peer-pressure involved. Overheard a conversation a while back that summarizes it nicely - someone was talking about a scheduling system written they've used for years, and mentioned it was written in Perl. Another participant guffawed, and after the requisite Perl-bashing, the original person allowed that, yes, even though it worked fine, they should rewrite it.

No idea what company that was, but I'd love to work in a place where that was the most pressing concern on my plate.

View on HN

I've encountered this sort of thing too many times here in the last few years. A set of Python scripts to ETL a few hundred MB of data once daily is suddenly desperately in need of replacement with a real-time, fault-tolerant MQ backed server (I've seen more than one of these, from custom ZMQ proposals to Kafka). A set of two or three scripts run on a compute server that are kicked off by cron and that do some work in batch on a modest set of data (few hundred GB) suddenly must be replaced with an asynchronous task management service capable of managing a huge set of processes with complicated dependencies (Jenkins, Luigi). A bit of data one team produces (once daily) has another team asking for it: instead of exporting the data from the DB directly or standing up an endpoint on, say, a Flask server there's a drive to slap a Spring microservice architecture in place. And so on. It's really a sight to behold, the rationalizations people go through who, in other contexts, laugh at the inferior logical reasoning abilities of non-CS folks.

Well...there's a lot of context in "works fine," from "it is perfect and there's nothing to add to it" through "good enough" to "it sort of works within most expected inputs, but everyone is afraid to touch it, so we've built a bunch of hacks around it."

Were all of these people competent?

Honest question. I have only seen the "it's written in X, so therefore it must be re-written in something nicer even though it is working" thinking from incompetent people who were just trying to take ownership of something they didn't quite fully understand. Though I never have seen it in a appropriately functioning commercial setting; if management is competent, they'll immediately recognize the high costs with no concrete benefit and say no.

It's one thing to say "we have to re-write this because it uses Java applets, and Java applets are problematic because Oracle is dropping support for them, so our customers are going to be screwed soon if we don't do something." It's another thing to say "we have to re-write this because it's in Perl because Perl is something I don't like."

To your last line -- I've got a PHP sub-system that I would love to have re-written to match our chosen stack language set, maybe simply because we could then have fewer required skills on the team (few of us are PHP guys). But for all its warts, it works. And that matters. I am thus hesitant to re-write for no other reason than 'we hate it'. It would be a lot of work for zero functional gain. (All benefits would be non-functional - and that stuff gets put on the back burner)

I've seen this situation multiple times, and yes the developers involved were competent. They were even well-meaning, and wanted to build something for the benefit of the company, not just their resumes.

I think the tendency to over-engineer and over-polish comes mostly from getting too invested in one particular project or task. The developers have "professional pride" - they want to deliver software that has good architecture, high test coverage, easy to understand and maintain code, reliable, scalable, etc.

This means competent developers are very tempted to continue working on a project as long as there are possible improvements to it, even if these improvements do not make business sense. Nobody wants to admit that "cron job that fails once per month" is a sufficient solution when they can see a better solution, and go work on the next hacky cron job instead.

Reply on news.ycombinator.com