Hacker News Reader

Comment by nkurz | Hacker News Reader

As someone excited by the improvements that are possible moving from AVX to AVX2, I was recently surprised to learn that Intel has chips released this year that do not support AVX. AVX-512 is still supposedly scheduled for release next year with Skylake.

How long do you suppose until it's reasonable to presume that most processors will support it? SSE 4.2 came out in 2008 and appears to be the baseline now, so 5 years seems like a reasonable guess.

It's hard to believe this degree of ISA segmentation is going to benefit Intel in the end.

View on HN

That's why I feel dynamic code generation is the future. JIT. It's madness to write high performance code to take CPU [1] and DRAM [2] specifics into account.

It's easy to write something that performs well on one system.

[1]: 32-bit or 64-bit -> how many registers are available. SSE2, SSE 4.1, AVX2, etc. Different prefetchers, cache configurations. Different instruction level limitations and performance. Differing amounts of cache with rather variable latencies.

[2]: Different number of memory channels, different distance between DRAM page changes.

Compilers generate static code that will branch to the correct code path based on your architecture. You don't need to JIT until there are so many paths your executable becomes too big.

Any references about this compiler?

icc does this, as an example. It got Intel into trouble because they changed the compiler at one point to use the slowest path on AMD processors.

https://gcc.gnu.org/onlinedocs/gcc-4.7.4/gcc/Function-Attrib... and http://pasky.or.cz/dev/glibc/ifunc.c

look up the target attribute and the ifunc attribute--it's basically a way to compile multiple versions of a function for different targets in a single source file and then use the dynamic linker to determine which one to resolve at runtime. obvious use is for things like optimized memcpy implementations.

So, it's otherwise automatic, except I just have to write a selector routine that tries to decide the best performing routine to run at runtime and implementation for each individual case with varying hardware support.

You only need to go to all that trouble if you want high performance across a variety of machines. If you are merely after bragging rights or trying to satisfy someone else's requirement, the theory is that you can compile the exact same piece of high level code using different optimization targets, and the compiler will do all the work for you, providing maximum performance for each instruction set practically for free...

Even more practically, Agner has a typically excellent description of the strengths and weaknesses of the dispatch strategies used by different compilers in Section 13 (p. 122) here: http://www.agner.org/optimize/optimizing_cpp.pdf

Well, you can write that code that tries to decide, or you can let GCC emit that code for you: https://gcc.gnu.org/wiki/FunctionMultiVersioning

All you need to do is figure out what's the best version of that function to write for that target, and let GCC do the rest of the heavy lifting.

Do the branches happen at run time or are the optimized routines selected at install time (or something else?)

At load time. gcc's approach is to substitute an entry in the PLT, the CPU dispatch logic being run at that time to select the contents of the entry.

I think that the SIMD instructions are well past the point of diminishing returns, both for the kinds of instructions provided and the width of the vectors. I'm more interested in how VT-d and TSX allow different software architectures, if and only if you can design your OS around the assumption that they're available. By only including them on select models, Intel's making them much less useful even for the customers who want to pay extra to have those features.

I used to write software rasterizers in a past life. AVX-512 is straight-up an observed 8x performance gain over SSE: a well-written software rasterizer is a very clever thread-scheduling algorithm wrapped around long lists of FMA-load/store sequences.

I know that ray-tracers, databases, and other 'big data' all gain equally-well from AVX-512. The IO-request depth on these new parts is such that, for ALU-heavy work, with good IO-spread, memory latency is completely hidden---fetch time is still ~250-300c, but you just no longer see it.

The only thing I miss is 'addsets', which was dropped from LRBni, as this instruction was 'the rasterizer function': it now requires a fairly involved sequence to replicate in AVX-512.

Pining for other lost things: if we had the LRBni up- and down- convert instructions, a software texture sampler would be a lot more feasible.

Does gcc support these new wider registers yet?

Yes and no. If you write the ASM you can use them, but it's code generation likely won't (but then I haven't checked for 2 months).

Most things that can be vectorizes will be placed in SSE rather then AVX. Also the GCC generally sucks at optimizing for SSE, or determining when code should use SSE as opposed to standard registers.

Generally speaking the LLVM backend does better SSE and vectorization code generation. But some think it used SSE to much/incorrectly.

So TLDR no

New wide registers are VERY new. The are barely supported, my processor has AVX2.0 and I have (still have to) one of these days set up perf properly because all of its fault codes aren't properly baked into the kernel yet (as of 3.17).

(Sorry for the lack of references)

Also the biggest draw back of AVX is they don't hold their state between context changes :/

Wider SIMD units may provide a linear performance increase for many workloads, but they have a super-linear impact on the up front cost of the chip, especially when you take into account the opportunity cost of those transistors: they could have been dedicated to something that might have also helped non-numerical workloads. Wide SIMD is great to have, but it doesn't come free, or else we wouldn't have GPUs.

It may just be that Intel has abandoned this use case to the GPU. Which obviously is bad news for those of us who still rely on CPU SIMD.

But wtallis' argument to the contrary is quite compelling: https://news.ycombinator.com/item?id=8801808

You only get those performance gains if you've got the cache and memory bandwidth - which is often very lacking.

E.g. AVX (8-wide) was added for SandyBridge, but it wasn't always that usable until Intel doubled the cache bandwidth with Ivy Bridge.

With ray tracing the increase in performance going from SSE (4-wide) to AVX (8-wide) for BVH intersection was only ~25-30% - instead of the theoretical 100% increase). You're generally limited by memory bandwidth.

Image processing, etc. stuff I sometimes write seems to benefit pretty linearly from wider SIMD registers, like in AVX2. It'd be useful to get up to a cache line wide SIMD registers. Cache line is the granularity memory subsystem works at anyways.

I wonder when cache line size will be increased from 64 to 128 bytes. Hopefully that wouldn't affect total number of lines, though. Mere 512 entries in L1D cache is already such a pain.

CISC is rapidly becoming a requirement, however.

Power requirements scale as the square of the frequency, and also, perhaps even more importantly, leakage power, and hence, heat generated per unit area, increases as the process size decreases.

As such, as process sizes shrink, less and less actual area of the chip can be active at any one time. ("Dark silicon"). In other words, you might as well have areas of the chip that aren't used most of the time, because most of the chip can't be active at once anyways.

I think that the multiple 32-way SIMD units on GPUs demonstrates there's plenty of room for widening on CPUs before we hit diminishing returns. The question in my mind is how much more like a GPU should CPUs become and Xeon Phi demonstrated how easy it is to get the mix totally wrong.

There are clearly plenty of common workloads that can make use of arbitrarily wide SIMD units. But nothing about the GPU market provides a clear indication that those SIMD units are better grafted onto the CPU than delegated to a coprocessor.

The programming model is certainly simpler if they're part of the CPU's normal instruction pipeline, but on the other hand most CPUs don't have the memory bandwidth to feed more than a handful of GPU-style SIMD units and certainly don't have the power delivery or dissipation capacity to handle the compute resources of a mid-range GPU in addition to the normal CPU.

Having the big SIMD units on a coprocessor card seems to be working just fine for the kinds of workloads that really need a lot of vector processing power. Those workloads also seem to not benefit much from things like speculative out of order execution, which frees up a lot of die space and power draw budget for more SIMD units. To some extent it's inevitable that SIMD code can't be making too much use of CPU design features that benefit branchy code, or else the code couldn't be vectorized in the first place.

The workloads that are making heavy use of SIMD on the CPU are mainly using it for bursts of number crunching where the problem is too small to be worth the latency penalty of moving it to the GPU, but that penalty is getting smaller with each generation. The small integrated graphics processors common on non-server CPUs can all but eliminate that penalty, further reducing the demand for extra-wide SIMD in the CPU cores.

Reply on news.ycombinator.com