Hacker News Reader

Comment by lukego | Hacker News Reader

I really enjoy this summary of the AVX512 instruction set evolution. It makes me curious to know more about the AVX512 microarchitecture evolution too. How practical is AVX512 likely to be over the coming years? How much do we have to worry about side-effects like throttling the CPU clock speed down when the wide datapaths are exercised, etc?

Can someone wake me up when it is a no-brainer to use AVX512 instructions on CPUs that support them? :-)

View on HN

I hope you have a comfortable bed, then. It will be a while before there are machines deployed across the whole range (client laptop, client desktop, server, network edge) that are SNC. Atom isn't even supporting AVX2 in the short term either.

Daniel Lemire, who I work with on simdjson, wrote this on throttling: https://lemire.me/blog/2018/08/13/the-dangers-of-avx-512-thr...

I don't know specifics (I'm ex-Intel a good bit before SNC, and wasn't a hardware guy) but my impression is the clock throttling impact gen-on-gen goes down. If you're actually getting good value from AVX-512 instructions rather than chucking in 1 in 1000 instructions for the cool factor, you're probably fine.

Thanks!

While you are here... :-)

What do you think about the tendency of glibc to use the latest available instruction set extensions for routines like memcpy? Should we be worried that when we call memcpy() in a program this will effectively mix some wide-datapath instructions into the CPU's instruction stream with potentially hilarious consequences like clock throttling? What's all that about?

As a library author, I am very, very nervous about doing anything that might cause clock throttling. After all, you will not see a 2x gain from twice the SIMD width (there are many other limiting factors, and of course, Skylake issues 2 x 512b operations max vs 3 x 256b ops).

Even if you did get 2x, you would have to balance that against clock throttling in circumstances where you have no idea of the _relative_ importance of your library. Are you 95% of the cycles, or 10%?

I'm always nervous about companies like Intel doing both hardware and software. If you do only hardware then you need to keep it simple or else software people won't use your stuff. If you are playing both sides then you can throw engineers at the problem and make it as complicated as you want. In this case they can just push complex AVX512 code into projects like glibc and bypass the whole "is this really worth the complexity?" discussion.

One related example is Intel CPUs having good cache-DMA integration but this not working across NUMA nodes. Ideally Intel would improve this feature to be NUMA-aware and everybody would be happy. However, it seems like they decided it would be cheaper to send forth an army of software engineers to put NUMA-scheduling kludges into a few key projects like Kubernetes and then call it a day.

/rant!

> good cache-DMA integration but this not working across NUMA nodes. Ideally Intel would improve this feature to be NUMA-aware

How do you propose they do this? You can't magically move the physical devices to the other socket, and transferring the data between sockets is what those "NUMA-scheduling kludges" are trying to avoid. The only solution is to have software put the data close to the device.

My take is that they should use the memory subsystem to route the DMA to the appropriate L3 cache.

I don't think the main issue is conserving QPI bandwidth: it's minimizing latency when a core loads the data into L2. That latency often causes ~30% performance drag on e.g. packet processing applications.

sometimes you get more than a 2x gain when going to a wider simd instruction set, because more instructions are available. This is especially true with AVX-512 and its masks.

IIRC glibc reverted those avx-512 string functions, precisely because it turned out that in most real programs outside microbenchmarks, the clock speed hit negated the benefit from wider instructions.

But IIRC, back when AVX was introduced, there was some issue with that as well, and similar with AVX2. But now those seem to have been mostly addressed. I guess in some years the same will happen with AVX-512 too?

Is there definitive information on these issues?

The pattern for me is that I'm excited about a new instruction set when it is announced, then disappointed when it is shipped in silicon, then I have a hard time knowing when it is truly resolved and worth taking a second look at.

My recollection with AVX2 is that glibc shipped those routines since before silicon was generally available, and then kept them enabled even when the clock-throttling issues became apparent. (The water was muddied a bit in one glibc release that shipped a bug in selecting which code to use for the running uarch, but that's a separate issue.)

Not specifically glibc but some data on the frequency scaling impact: https://blog.cloudflare.com/on-the-dangers-of-intels-frequen...

Re AVX-512 and throttling, may I suggest this post instead https://lemire.me/blog/2018/09/07/avx-512-when-and-how-to-us..., again by Daniel.

Reply on news.ycombinator.com