Hacker News Reader

Comment by dzdt | Hacker News Reader

Twenty years ago assembly language was losing its last battles with higher level languages as a viable choice for game development. Skilled assembly language programmers insisted they could give a factor of two more performance on the same hardware. It wasn't enough: everyone moved on.

Now the author is again saying his SPMD hand-coded assembly language can outperform higher level languages by a factor of two for some test cases.

I don't see the economics working out any better for the assembly language solution this time than last time. It won't port across different hardware generations, is hell to write and maintain, and requires super-specialist knowledge.

What this might point to is room for a new high-level language that maps better to the reality of current hardware.

View on HN

To be fair, Java programmers sweat annotation bullets to cajole the JIT to make Java code perform to within 50% of equivalent C code so getting this kind of performance from the get-go seems like a pretty good deal to me.

See also maxas versus CUDA in Nervana's Neon library on github. In this case, naively, it's used to make SGEMM operations that are 20-30% faster, but across the entire universe of SGEMM inputs, it can be 3-6x faster. Whether this is because of the use of assembler or because Scott Gray is a better dense matrix programmer than the current cuBLAS team is an open question because cuBLAS is closed source.

Finally, you've ignored the 800 lb gorilla in the room: Intel is making processors that look more and more like GPUs in terms of cores, vector units, and SIMD width whilst investing in FPGAs through the purchase of Altera. But what they're not making is a consistent paradigm for programming them. They could embrace an open standard like OpenCL. They could give NVIDIA a flaming middle finger and build a CUDA compiler like AMD did (Rhythm). Or they could make one of their proprietary languages like Intel TBB (https://www.threadingbuildingblocks.org/) or ISPC (https://ispc.github.io/) a first-class citizen and throw their massive support engine behind it. But instead, they seem to throw these APIs over the fence and to continue to rely on "recompile and run" in the hopes that optimizing compilers can magically parallelize otherwise serial code.

Which is why I continue to stick with CUDA/NVIDIA...

  What this might point to is room for a new high-level 
  language that maps better to the reality of current 
  hardware.

Isn't that exactly what the author said in his last two paragraphs, including a hand-wavy sketch of what he thinks that high-level language might look like?

Since the site is down and I can't read the article I can only guess here.

For me it's something with value types & memory coeherancy sematics. That's what you need to get the most out of your hardware.

Good candidates for that today are Rust, C, C++ and maybe C#(don't have enough experience with their value types to make a judgement yet).

Surely you're not suggesting this option should not exist at all for who need this (or at least think so)?

For games, it might be about your game running at 30 fps or 60 fps.

If your future server farm is projected to cost $20M, wouldn't you want to try something like this to save a few millions?

> I don't see the economics working out any better for the assembly language solution this time than last time. It won't port across different hardware generations, is hell to write and maintain, and requires super-specialist knowledge.

Sometimes that's the price you have to pay. Such is life.

That was done with C. It was designed to run on the PDP-11, and was a very good fit for that particular architecture.

It would be better if hardware manufacturers designed their CPUs around high-level languages, not the other way around. Burroughs did with their mainframes, which ran Algol 60 natively and also had instructions for efficient COBOL execution. Later, Lisp machine manufacturers made machines specifically designed to run Lisp.

When hardware is designed without reference to the software which later runs on it, never mind the problems the software is designed to solve, it often leads to solutions tailored better for the problem, but which don't fit the hardware particularly well, not being attempted.

Many if not most problems involve processing different data, usually non-numeric, in different ways. MIMD hardware would be better for that.

Designing around languages only works in a world where you have an infinite cache size. Most compilers/JITs are great at inlining. Where you run afoul of things is programs jumping around in memory and cache missing all over the place which is just about every high level language.

Unless you have explicit value types and memory coherancy guarantees then you'll always see this tension.

It's not like hardware vendors set out to build slow chips or don't know about high-level languages, it's just that hardware doesn't work that way.

So the choices are:

(1) Hardware designers need to do much more to speed up general memory accesses, so that a CPU cache is no longer needed.

(2) Programmers need to accept that if they use pointers (either explicitly like in C, or implicitly like in Lisp or Java), they will have to tolerate the performance hit.

(3) Programming is made much more difficult by requiring programmers to take into account memory layout, which is almost like a return to the bad old days when you had (by today's standards) tiny memories.

(4) Memory is distributed over multiple CPUs, each working on a different part of the problem.

There aren't many problems where users (who today, are almost everybody) care about raw speed. What they do care about are response times.

I'll add 5:

Split application into logic(branches, special-case, and all the gnarly stuff that CPUs don't like) and data processing(nice structured data that fits well into memory). This is essentially what Sony did on the PS3, and made for a very fast system. They just did a poor job of messaging it ahead to devs who were familiar with n-core systems.

That way you can speed up the things that need to be fast and the slow stuff doesn't interfere with it. We aren't going to see any fundamental shifts in memory tech so I think this is the best mitigating solution.

Relational Databases already kinda nail the structured part to a degree, I think where we fall over is mixing logic with the data we want to process.

There's also a continuum of prototyping -> high performance that gets missed in some of these discussions. If I want to see if something works I can knock out a quick prototype in JS but if I want to ship a performant, solid product I'm going to use Rust/C++/etc.

20 years ago, CPU performance was doubling every 18 months or so.

A computational chemistry program I know uses at least four external libraries which rely on assembler for their kernels (BLAS, FFT, small matrix multiplication, and scalable matrix diagonalization). That provides a factor of several over Fortran, often running on petascale systems. The economics work in such cases, at least; Fortran-callable scientific subroutine libraries are one of the more successful examples of code re-use.

Reply on news.ycombinator.com