Chunking Optimizations: Let the Knife Do the Work
nullprogram.com
nullprogram.com
The main use cases for vector intrinsics, in my opinion, are when you are doing something clever shuffling the vectors around or packing / unpacking them. If you’re not being especially clever, just let the compiler do the heavy lifting and make it happen by putting "restrict" in a couple places.
If you are being especially clever, at least give the compiler a shot and benchmark the different versions against each other.
My personal projects are are beautifully simple loops that compile to vectorized AVX or SSE instructions. My work projects have a lot of `//todo: rewrite this with intrinsics` or `//don't fuck up the alignment there's a bunch of SIMD intrinsics under here`.
xor512d:
vmovdqu8 zmm1, ZMMWORD PTR [rsi]
vpxorq zmm0, zmm1, ZMMWORD PTR [rdi]
vmovdqu8 ZMMWORD PTR [rdi], zmm0
vzeroupper
ret
which is very close to the reported Clang-generated code.Flags used: -O3 -mavx512f -mavx512cd -mavx512bw -mavx512dq -mavx512vl -mavx512ifma -mavx512vbmi
Not it doesn't. The compiler chooses how many bytes to do at a time. And in fact it defers that choice to run time. For reference, here are both functions compiled with GCC and Clang at -O3: https://godbolt.org/z/WmduxJ
For xor512a both compilers insert an aliasing check and fall back to a scalar loop if the input arrays overlap. But for the case where they do not overlap, both compilers produce vector code that is essentially equivalent to xor512d.
If we expect aliasing to be rare, the two functions will usually both execute essentially the same vector code, with the performance difference only being the slight cost of the aliasing check. This holds for both GCC and Clang.
Let's say I build a binary with AVX support. Will it run on a CPU without AVX support?
It's less of an issue with older SSE stuff that is effectively universal across all X64 chips, but it's a big issue with late SSE and AVX extensions.
There are a couple relevant GCC flags: -mcpu and -mtune. The -mcpu flag will specify which features it is allowed to assume are present. The blog post is using SSE instructions, I’m thinking SSE2 but that’s just from memory, and these days most people consider it eccentric to try and support CPUs without SSE2.
There are a couple ways you can do the feature testing in your own program. You can make multiple copies of the function and compile them with different CPU targets, and then use something like
void xor512a_avx(...) __attribute__((target("avx"))) { ... }
if (__builtin_cpu_supports("avx")) {
xor512a_avx(dst, src);
}
Alternatively, you can use something called function multiversioning, which will write the branch for you. I think this feature is present in both GCC and Clang.Also note that there are some unusual edge cases you may not have considered with feature detection. There’s a strong case to say that instructions like CPUID are the wrong way to go about it, and you should do things like use sysctl() on macOS. The risks in using CPUID are a bit esoteric in my opinion, though.