Hacker News Reader

In rewriting projects, I often find that as I rewrite them, I can knock out requested features along the way. I've taken this approach when fixing up some of my old personal codebases. It's a two birds one stone approach; it takes me just as long to fix code in the old framework as it does to rewrite the code in the new framework without the buggy behavior, so where possible I do that.

We had a similar practice in the restaurant world before I found work as a programmer, and we called it "clean as you go." I think it's quite appropriate. The bug reports will always be there, the feature requests mountainous, and yet the messes don't clean themselves. There must be balance.

Do what needs to be done, but make a point to improve your codebase with every new change. Prefer consistency over new hotness, and ease of maintenance over unreadable micro-optimizations. Be a positive force in your codebase, and future developers, especially including yourself, will appreciate your efforts.

I agree with "clean as you go", but I'm very cautious. Rewrites are scary and risky in legacy code. Usually I'll add a few unit tests to increase coverage and de-risk the change I just made.

Leaving code as is can also be scary and risky. Being asked to overhaul an existing system on day 1 is scary though.

I tend to hold off on rewrites until I can picture the new system - complete with a solid understanding of exactly how it will fix the problems of the old one. And yes, with good tests.

Aye, I agree completely. When approaching a given rewrite, I'm very hesitant to go through with it unless I have a strong understanding of all the files I'm about to have to touch to fix the resulting compiler errors. Ideally this should be straightforward, but the tight coupling often observed in poorly maintained codebases can cause the most innocent of small changes to spiral out into the template metaprogramming circle of death.

Even then though, all is not lost! When faced with an architecture change that really should be made, but is too large to make right now, I make a note of it on a project board somewhere and now I have a new sub-task. Any time I touch code elsewhere, I work to reduce coupling and correct dependency issues, so that the original change is easier to make. By slowly cleaning up small parts of the code, I open the door to making the larger rewrite not quite so large when I get around to it, and hopefully improve the project's overall quality in the process. In this way, I create forward progress towards the end goal without needing to commit to all of it at once.

I think this epic rant against rewrites still is mostly valid 17 years later:

https://www.joelonsoftware.com/2000/04/06/things-you-should-...

It was wrong then and it is still wrong.

First of all Netscape had the problem of incorporating opaque blobs from other sources. When they went open source, they couldn't keep those so ripped them out. Then the code didn't even compile.

Secondly that Netscape release that he mocked? That was the start of Mozilla, Firefox, and so on.

Third, how did Netscape survive? My impression from the sidelines is that it was a negotiation ploy from AOL. When their IE contract was up, they needed a viable alternative. In the end Microsoft let them continue using IE, PAID THEM MONEY, and had them shut down Netscape. They were laughing about this all of the way to the bank. And then Mozilla ate IE's lunch.

There have been a lot of rewrites. Some succeed, some fail. But it isn't always a bad idea to do one. And Netscape was a necessary one that succeeded in every measure that it was supposed to.

You are being charitable to Mozilla (the application suite). That was a bloated, buggy mess for years. I recall 2001-2003 was a really bleak time to be a Mozilla user. Firefox launched as a response to Mozilla and was more lean, but it wasn't until 2007 or 2008 that HTML5 features started trickling in and Firefox started to gain marketshare and developer mindshare for the advanced features it could offer.

And that was when Chrome showed up. So from 1998-2008, Mozilla had an open season to gain ground on IE, and their marketshare only ever got as high as 30%. It was 2011 or 2012 that Chrome was stable enough and Google started their upgrade-to-Chrome adverts on google.com. And then Firefox faded back into it's 10% niche. I'm a developer, and my peers look at me like I'm a weirdo because I favor Firefox and do my primary development on it. That shouldn't have happened, Firefox should not be niche given the time they had to work.

It is inspiring to me that the Mozilla project didn't fail and got so much love, but starting with that original Netscape codebase and the Gecko and XPCOM designs really hindered them. Mozilla missed out on offering a competing engine for Node.JS because their JS engine is so tightly coupled with the rest of the browser(?) - that's just one huge opportunity missed. Offering a competitive mobile OS at the advent of smartphones in the vein of WebOS is another, albeit less egregious, miss. The codebase just wasn't clean nor adaptable enough to break free of the desktop.

It wasn't until 2007 that HTML5 features began trickling in anywhere...

Right, and what I was alluding to in that point was, why should any user switch from IE if it has feature parity with Firefox or any other browser?

Firefox had developer tools in Firebug, so it was already getting a geek user base, but when Firefox got some legitimate new features like Canvas that other advanced browsers like Safari had, regular users began to dabble. With the occasional news report that honestly reported the security nightmare that IE was, I found that between 2006-2009, Firefox was an easy sell to regular users.

The way I see history, Mozilla floundered between 1998 and 2005 and had almost no marketshare. All the Netscape rewrites that came before were a shitshow and the AOL thing must have been a huge (but necessary for funding) distraction. Probably the only fair interpretation of history is that, of course, the codebase was a mess given the development practices back in the day.

That said, I recall the Gecko rewrite was a long and painful one. Was it unnecessarily so? I believe it had a lot of dumb component architecture concepts and I would love to understand what the drivers were there. Did they believe that components were the best way to organize a huge open source project? Woulda, coulda, shoulda thinking about Mozilla makes me really sad sometimes.

> their marketshare only ever got as high as 30%

You write that like it was a failure. It was HUGE! In the context of the ms monopoly, it was an unbelievable and largely unmatched success.

Secondly, having a third browser overtaking it was not a failure either, but success in the goal of providing viable alternatives to the stagnating ie.

Remember that Netscape started out dominating this market.

Maybe the rewrite didn't cause the catastrophic loss of marketshare (I'm going to resist my urge to spend the morning researching this), but it's hard to see it as a great success in that context.

Netscape's fall had many causes, but the biggest one was monopoly actions by Microsoft. Microsoft got sued for this, and lost. The fact that Mozilla crawled back to significant marketshare from there was unbelievably impossible. The fact that open source efforts got to the point where the web became usable again on Linux was simply amazing.

Microsoft had checked out of the browser war game in the 2000's, they had won.

All of the most important Mozilla improvements in software design, development practice changes, and testing methodologies in Firefox came in response to and after the arrival and ascendance of Chrome, not IE.

Mozilla was not destroyed by Microsoft, and Microsoft gave Mozilla ample time and ammunition to be disruptive.

I know you want to be positive on Mozilla, I want to too. But the overall program has been kindof a failure in terms of mindshare.

Mozilla abandoning the addon ecosystem left very little reason to use firefox over other alternatives.

I think this kind of sourpuss attitude about Mozilla isn't helpful. What they did with addons is about maturing the platform, making it a safer operating environment for non-power users.

I believe it's a matter of time and maturity for Mozilla to support 99% of add-on requirements with their more secure, chrome-like, addon system. The fact that vimperator can't be written in today's Firefox, or that it would be a mighty pain to do so right now, is acceptable to me if it means Firefox is less hackable.

I've certainly been critical of Mozilla and dislike how the browser has been dumbed-down in many respects over the years in-service of making it an application delivery platform. But, in my opinion, stability and security have to be job #1 for Mozilla.

It would be great if the old world of XUL/XPCOM was retired after the new way of doing things could cover the same range of use cases. Breaking userspace without lining up alternatives is why users are up in arms.

Yes, holding off on e10s transition until feature parity for the most extreme popular plugins + a reasonable amount of time to rewrite would have been the most savvy thing for them to have done.

When Mozilla started, it was not clear that there could ever be a browser that would take marketshare from IE. Common wisdom was that the browser wars for over..for good.

We actually got to the point where 30% of people who bought a computer went and installed a different browser. That's amazing. How many computer users don't have a clear understanding of what a browser is, let alone how to install one?

Before Google blindsided the industry, nobody believed that anyone would ever put anything like the effort that they did into improving JavaScript. I cannot criticize anyone for making decisions that seemed perfectly reasonable at the time but didn't age well.

As for HTML5, there was a chicken and egg problem. When you don't have marketshare, nobody cares about your wonderful features that nobody is using. So you have to get feature compatibility first, then marketshare, and only THEN turn to wishlist items. All that Microsoft needed to do to keep the browser wars won was keep investing in IE. Make it a moving target that nobody can catch up to.

They didn't simply because they had won. Won so completely that they thought that they could rest on their laurels.

> you probably don’t even have the same programming team that worked on version one, so you don’t actually have “more experience”. You’re just going to make most of the old mistakes again, and introduce some new problems that weren’t in the original version.

What? This statements are very arbitrary.

It isn't really when you consider the context. Many times when people feel like doing a re-write, it's because they don't understand the old system, or the people who wrote the old system are no longer around. Otherwise... why would you need to do a single big re-write? The old guys could just fix it up. Or the new guys could spend more time reading code.

The exceptions to this would be if you were motivated to do the re-write for a good reason, like it depends on something that is no longer dependable (e.g. Java applets or Flash).

This is in line with Naur's "programming as theory building" essay [1].

Is the system "alive" or "dead"? The system is alive if the programmers that are maintaining it have the theory of the system in their heads, and can use it to inform change, and can teach that theory to others.

The system being alive is not a property of the system in isolation but a property of the relationship between the system and the programmers who work on it.

[1] http://pages.cs.wisc.edu/~remzi/Naur.pdf

Broadly I agree, lately though I've had to change some stuff that was tightly coupled to only slightly related code via generics and inheritance. Rewriting parts is the only way to disentangle things. If there is one thing that spaghetti programmers love it's inheritance.

The Boy Scout Rule[0]

[0]: http://programmer.97things.oreilly.com/wiki/index.php/The_Bo...

Clean as you go sure, but in a twisted and janky codebase, full of absurdist abstract factories and eerie hall-of-mirrors template meta-programming, you have watch your step.

Deep, deep in the code there are bugs which have lain undististurbed for eons. Scrape away some cmake pus, sprinkle in the mandatory security fixes, sure no problem. But dig to deep with the 'WTF were they thinking reimplementing the preprocessor?' shovel and you'll wake the helter-skelter bugs from their heisencave into reality.

Also, you mustn't awake the Balrog. The possibly bearded old programmer, he used to be a Hero programmer, a mythical 10x coder, and he thinks he owns ALL THE CODE. He will revert your SCCS checkin and put the GOTOs back in the code.

We share a similar mindset. I actually get a kick working through crappy code as I find reverse-engineering quite exciting, in a kind of "order from chaos" way. Take something crap, make it better, learn something and get paid for it. Ideal!

As someone who usually writes the crap stuff that other people get to live with, now I'm on the other end and having to make this "order from chaos". It's one of the most difficult things I've ever done.

I just hope I can figure out how to enjoy it, eventually.

At this stage the only thing that keeps me going is I'm too proud to feel ineffective and to fail at this, ugh!

Chin up! Here are some tips:

* Reframe the task as a game. A crime to solve if you like.

* See it as a series of black boxes that you keep subdividing (You are working top-down right?)

* Keep a [cork|black|white]board like they do in the movies with all the suspects and details of the case, with the string linking them (Do this digitally if you like)

* It's more important to know the I/O of the boxes/components than how they work, at least initially

* Be transparent with your employer about the difficulty. They know it's a mess - that's why you're there.

Good luck!

Nothing teaches you how your technology of choice really works like taking apart a massive horrendous codebase.

You'll be at the next level of expertise soon!

I often get more satisfaction from removing code (and keeping / improving the functionality) than I do from writing new code.

Maybe I am weird but I like doing stuff like that. I am more of an optimizer than from-scratch-creator so I like having a starting point. Then you just refactor your way through the system while adding new stuff and fixing bugs. I also like debugging and profiling :-(.

Nothing wrong with that! There are more of out there than we think :-)

Can i hire you both?!? Seriously, are there many people that relish refactoring? I'd like to meet them. How to find them?

Seems you can find them here :-)

I've been thinking about marketing myself as "Code Janitor", actually :)

If you have any Django / Python based stuff that needs refactored and are willing to hire remote I would be up for it.

Plenty of us out here!

Here! I spent a lot of my early career porting legacy code from one language/platform to another and I liked that well enough. Now I spend more time refining my own code than is really necessary, just because I enjoy it.

Make me an offer! :-)

I'm with you. The worst part of a project (when I was doing them) was getting the first to end-to-end proof of concept going.

Give me profiling tools, debuggers and refactoring tools any day of the week.

The elephant in the room here — and this is no judgement — is that the contractor isn't worrying about creating more technical debt or something that will fall down in a year's time. It's YAGNI taken to an extreme. As easy it is as a contractor (as I can attest) to make fun of previous implementation spacemen who came before you, they often had the laudable goal of creating something that would be easier and cheaper to work with moving forward. I've also been at - more than once - given managerial control of a large codebase driven exclusively by the need to deliver deliver deliver and had to explain to the company why development has become so much slower since they started.

> As easy it is as a contractor (as I can attest) to make fun of previous implementation spacemen who came before you, they often had the laudable goal of creating something that would be easier and cheaper to work with moving forward.

It's one thing to have the domain expertise to accurately anticipate future asks and preemptively prepare for them. But I've seen very few cases of that, compared to projects that were design pattern lasagna. Layer upon layer of abstraction to solve imaginary problems that were never real nor will ever become real. And all those abstractions cost the same as those which actually solve problems.

My approach is always to solve the problems you actually have, and no more. This includes abstractions for unit tests; that's an actual problem. (You are putting all those external dependencies behind interfaces for mocking, right?)

Yep, the daily work cycle at enterprise consulting.