Hacker News Reader

Comment by phillipcarter | Hacker News Reader

"roll those loops back up and don't compile in debug mode to achieve significant performance gains" is not the takeaway I thought I'd have going in, heh.

View on HN

Unrolled loops trade instruction cache for cycles. Whether it's good depends on what your bottleneck is, in this case ram access is a bigger bottleneck than cpu due to being shared with the rendering coprocessor. I wonder if we'll hit this crossover point again in the future.

And the code N64 was particularly unrolled for whatever reason. IIRC, there was one game that bit me when writing a JIT N64 emulator because there was a single code stream with no branches that was larger than multiple MMU pages. My JIT kind of assumed that actual code in practice wouldn't be larger than the actual N64 I$ without branching. Womp womp.

But hey, I guess I don't want to be judged on my mid-90s code either.

From what I understand (as I didn't start writing code until the mid-2000s), loop unrolling was a very highly regarded optimisation of the early 90s.

And it was a very long time before programmers (and compilers) realised just how bad of an idea loop unrolling was once we entered the era of superscalar processors with instruction caches and half-decent branch prediction.

It never occurred to me that loop unrolling could end up being a de-optimization due to compiler optimizations!

The things the parent comment referred to (instruction caches and branch prediction) are in the processor hardware rather than other compiler optimisations.

To be fair early 90s branch prediction was garbage. Back then it seemed like every new processor release was accompanied by a note on how branch prediction was improved by 80-90% compared to its predecessor.

To be even more fair, counted constant sized loops never required particularly good branch prediction, especially if the code is well laid out so that your "miss" does literally one more cheap step.

Misprediction there means you computed one more vector or value, not a huge loss compared to a cache miss. Branch misprediction hurts a lot when it forces a cache miss mostly, whether it's for data or for instructions, or ties up a deep instruction pipeline completely with lousy abort, which most CPUs do not have. (See: Nehalem.)

> And the code N64 was particularly unrolled for whatever reason.

My naive assumption is that they performed the majority of testing with debug code (as you would). And, since they had exactly one chance to get things right (being a physical cartridge), they couldn't afford to do all the testing again, with the new/different binaries generated with debug turned off.

My understanding is that the massively unrolling loops thing had to do with how different branches felt versus previous consoles. Compared to filling an I$ line (somewhere in the many dozens of cycles on an N64), a branch mispredict was very cheap. However branch mispredicts were very visible. Delay slots were present in the ISA to decrease their cost and the pipeline stalls were noted in pretty much all literature on RISC at the time. On top of that, the N64 really didn't have great performance monitoring tooling, but was complex enough of a memory hierarchy to have really benefited from such tooling had it been available. It's pretty hard to understand the system effects pieces of code have without that.

He did so much more than that. The not compiling in debug mode was well known before this video, and he addresses that. His speedups are independent of not compiling in debug mode.

I know, and I hadn't appreciated all the ways that doing something less CPU-efficient was actually faster due to the way the hardware was architected.

I just actually shouted when he said that if you simply compiled for release mode it ran 30% faster.

It was common belief in that era that you made code faster by making it longer and more complicated. The opposite is true now (because of cache size and deep OoO) - simple predictable loops are great - but even then people tended to overdo it.

Of course, it's a way worse problem that they compiled at -O0.

>Of course, it's a way worse problem that they compiled at -O0.

There was a Modern Vintage Gamer video going into this. Apparently it wasn't a mistake or misunderstanding, but a bug that existed in the original version of the compiler that caused issues when compiled with optimisation. Even then, the main library that did most of the heavy lifting was compiled with o2.

Super Mario 64 was a launch title, and they simply didn't have the time to work out the kinks in the compiler to get those few extra fps in Dire Dire Docks.

Yeah it's hard to remember the time before you could get 2 near-perfect C/C++ compilers on the 5-ish most popular architectures.

And the architectures have consolidated since then.

As I recall, that MVG video was incorrect—Nintendo could have compiled with optimizations and the game would have worked.

The most plausible explanation to me (and, IIRC, the one that Giles Goddard speculated was the case) is that they simply didn't want to take the chance that optimization could surface unknown bugs in the code, and they knew that the unoptimized version worked and played fine, so they played it safe and shipped the known-good unoptimized version. This is a perfectly understandable decision in terms of risk management, even if in retrospect it was too conservative. Less likely, but still very plausible, was that turning off optimization was just a mistake; mistakes happen when you're rushing a release for a deadline, and build deployment was not as orderly and automated back then as it is today.

Sounds like a good call to me. If you have something mission critical that's tried and tested, and reliably does 99% of what you need it to, you don't mess with it. It's very easy in software dev to get caught up in software optimizations, and it's a lot harder to quantify the risk of continuing to mess with something the closer you get to ship date.

Of course it "shouldn't make any difference" but who among us hasn't at some point been bitten by a "that couldn't possibly affect... ohhh. Ohhhhhhh." style bug?

When you're dealing not just with a single piece of software, but an entire hardware+software ecosystem, you don't risk jeopardizing the whole stack for an extra few FPS in some back corner.

Especially in an era where a buggy release could require an expensive return/reprinting step.

Back in those days it had to ship as close to perfect as possible because you couldn’t just send out a patch layer.

-O2 was considered "OK-ish". With -O3 often you get a can of worms.

Yeah, and most development houses would test and develop with -O2 on. The problems came if you ended up thinking you might need to switch near the end.

No, that's not more plausible. Because it's only the NTSC release that was compiled in debug mode. The Japanese and PAL releases were O2

But those are different versions. Just because they worked with -O2 doesn't mean NTSC would.

I'm not just saying this in a theoretical sense either; I've seen much weirder bugs.

It's always been a mix. Inlining, unrolling, and otherwise choosing longer sequences of instructions that are faster is still a major theme of optimization and optimizing compilers.

Even on today's high performance CPUs, which are far more sophisticated than the primitive 5-stage scalar in-order R4300 (not sure it even had any branch prediction actually).

> Even on today's high performance CPUs, which are far more sophisticated than the primitive 5-stage scalar in-order R4300 (not sure it even had any branch prediction actually).

I believe it didn't, with a short scaler pipeline (only 5 stages) the cost of waiting for a conditional branch to be calculated is relatively less, and MIPS of that era actually exposed the idea of a "delay slot" - rather than pausing the pipeline while calculating the jump, it still executed the next few instructions - so they were always executed no matter if the jump was actually taken or not. Filling this with NOPs makes it equivalently the same as a pipeline bubble, but I guess the intention was in some cases it could actually be used for something.

I think a few RISC architectures of that era did similar things, their entire goal was simplicity after all, the argument being that something like a branch predictor and the logic required to reverse speculatively executed instructions "should" instead be used to make the CPU smaller or higher performing in the best case (either as implementing better performing but larger logic, or higher frequency due to shorter critical paths).

I think this was dropped in later ISAs as it confused a lot of people, it wasn't well used by compilers (which is interesting as much of the MIPS ISA was explicitly designed around what would be easy for a compiler rather than handrolled asm), and improving process technology made transistors to handle the expensive superscaler speculative execution architectures relatively cheaper.

Reply on news.ycombinator.com