Hacker News Reader

Comment by amelius | Hacker News Reader

In the 90s we had segmented memory programming with near and far pointers, and you had to be very careful about when you used what type of pointer and how you'd organize your memory accesses. Then we got processors like the 286 that finally relieved us from this constrained way of programming.

I can't help but feel that with CUDA we're having new constraints (32 threads in a warp, what?), which are begging to be unified at some point.

View on HN

While reading I thought you were going to suggest unified memory between RAM and VRAM, since that’s somewhat analogous, though that does exist with various caveats depending on how it’s setup & used.

SIMD/SIMT probably isn’t ever going away, and vector computers have been around since before segmented memory; the 32 threads in a CUDA warp is the source of its performance superpower, and the reason we can even fit all the transistors for 20k simultaneous adds & multiplies, among other things, on the die. This is conceptually different from your analogy too, the segmented memory was a constraint designed to get around pointer size limits, but 32 threads/warp isn’t getting us around any limits, it’s just a design that provides high performance if you can organize your threads to all do the same thing at the same time.

You can blame ARM for the popularity of CUDA. At least x86 had a few passable vector ISA ops like SSE and AVX - the ARM spec only supports the piss-slow NEON in it's stead. Since you're not going to unify vectors and mobile hardware anytime soon, the majority of people are overjoyed to pay for CUDA hardware where GPGPU compute is taken seriously.

There were also attempts like OpenCL, that the industry rejected early-on because they thought they'd never need a CUDA alternative. Nvidia's success is mostly built on the ignorance of their competition - if Nvidia was allowed to buy ARM then they could guarantee the two specs never overlap.

CUDA clobbered x86, not ARM. Maybe if x86’s vector ops were better and more usable ARM would have been motivated to do better.

Whole concept sounds like groping in the dark for a Take to me: GPUs (CUDA) are orthogonal to consumer processors (ARM / X86). Maybe we could assume a platonic ideal merged chip, a CPU that acts like a GPU, but there's more differences between those two things than an instruction set for vector ops.

Yeah, that’s true. CUDA is in large part for big HPC servers, where ARM historically wasn’t a player and still isn’t dominant. x86 got clobbered for HPC by CUDA.

ARM has SVE these days. This comment makes no sense, anyway: people don’t do numerical computing on phones.

I bet the majority of AI inference FLOPS will be executed on phones before long.

Our phone camera pipelines are doing lots of numerical compute already.

> Since you're not going to unify vectors and mobile hardware anytime soon

Apple's M4 as described in its iPad debut has about the same chip area for CPU and GPU/vector functions. It's as much a vector machine as not.

https://www.apple.com/newsroom/2024/05/apple-introduces-m4-c...

I'll believe it when autovectorization is actually useful in day to day high performance coding work.

It's just a hard problem. You can code ignorantly with high level libraries but you're leaving 2x to 10x performance on the table.

386?

Reply on news.ycombinator.com