Hacker News Reader

Comment by 0x0203 | Hacker News Reader

The OS I work on currently supports a maximum of 254 logical cpus. If we ran on the 9754 and had to ignore two entire threads, we'd be giving up a maximum of 0.78% performance. I remember being excited about the first dual core systems not that long ago... Makes me wonder how long this trend of just packing more cores into the chip can continue, and where it will go after that. There seemed to be a fundamental shift between the Pentium 4 era and multi-core. Then between 8 core and many core. Big/Little also working its way in there. What's the next major innovation that's going to drive this ever-increasing performance limit? Or is there a theoretical maximum a single chip can do?

View on HN

> Makes me wonder how long this trend of just packing more cores into the chip can continue,

Latest GPUs have 10,000+ CUDA Cores, so, if you can paralyze the work, evidently pretty damned far.

Of course we'll need ever faster versions of PCI-E to come out to feed these beasts with enough data they don't sit idle most of the time.

The cuda cores of Nvidia GPUs are closer to fp32 units in vector ALUs than CPU cores capable of operating independently in parallel. Following that definition a modern CPU core would have dozens of "cuda cores" as well (although far fewer than GPUs optimized for that kind of workload)

More comparable would be the ~130 streaming Multiprocessors of a H100.

The fact that each cuda core also has its own instruction pointer is quite misleading. You would think that this lets it run different instructions per cuda core but the opposite is the case. The driver uses these instruction pointers for finer scheduling granularity. That is cool but is not the same.

https://stackoverflow.com/questions/58071834/why-does-each-t...

In the NVIDIA peculiar language, which seems to have been chosen with the only purpose of causing confusion, because all the names invented by NVIDIA are applied to things that already had traditional names for many decades, warp means thread and thread means SIMD lane.

NVIDIA has never given a good explanation about what they mean by the "instruction pointer" that belongs to each NVIDIA "thread". It certainly does not mean what in means normally, i.e. a special register that contains the address from where the next instruction will be fetched for execution. I believe that this "instruction pointer" refers to a register where the actual instruction pointer is saved when a "thread" is stalled because it has diverged into two branches after a condition test and only one of the branches continues to be executed, while the other branch must be executed later, with the complementary predicate.

These saved instruction pointers are presumably used for scheduling the "threads" to be executed by the SIMD lanes provided by the hardware, in such a way as to satisfy the cross-lane dependencies.

> Or is there a theoretical maximum a single chip can do?

There's a point where you won't be able to pack more transistors on a chip, but if Cerebras' work is an indication, chips themselves can get quite large.

But remember these chips are not designed with desktop users in mind. These chips are more similar to the ones inside very large computers that are built to host hundreds of simultaneous virtual machines on behalf of hundreds of users.

Sadly, we won't see desktops based on these unless we custom order them.

Linking a quick intro to these 850000 core beasts: https://www.anandtech.com/show/16626/cerebras-unveils-wafer-...

And STH recently posted an article about how they're used: https://www.servethehome.com/100m-usd-cerebras-ai-cluster-ma...

> Makes me wonder how long this trend of just packing more cores into the chip can continue, and where it will go after that

It really depends on the cache and memory architecture. You need to be able to feed data to the CPU efficiently.

And close to 50% on a double socket with 512 logical cpus ;)

Reply on news.ycombinator.com