Comment by vvimpcrvsh | Hacker News Reader

Comment by vvimpcrvsh | Hacker News Reader

Parent Full thread

I'll add benchmarks soon; I'm trying to make it usable first ;).

Right now, it's within 5% of ugly-style explicit SIMD code in the absolute worst case (the example in the README is more like 1%). The hot bit (the body of the map) compiles down to 100% simd instructions in the order LLVM thinks will be the fastest, and the load/store bit has an extra branch per load/store. This is because faster is mainly a bunch of 1-line wrappers around SIMD intrinsics, and a bunch of polyfills which only get used on machines which don't support those intrinsics.

The main reasons you'd experience a perf hit compared to explicit SIMD when using this library would be using runtime feature detection (minimal, and not implemented yet - would add around 2 branches per simd_iter call), getting lazy with gathers/scatters rather than ugly bit-level hacks, and accidentally using operations which aren't vectorizable on your machine (scatters on non-avx512, reductive operations on non-sse4.2, etc) and not realizing it (explicit simd will throw a SIGILL; faster will use a scalar polyfill).

Can't you test for the capabilities of the processor once and use the right SIMD code from then on? Maybe it makes sense to introduce a block around the SIMD code, which gets compiled into different blocks of SIMD instructions. Then you only pay for choosing the right block once. That's what I would do in Haskell at least.

This has to be done at a granularity coarse enough that the function call overhead doesn't kill the performance. It has to be entire loops, for example, not just some primitive operation like a dot product (which has at least 3 different implementations on x86).

On my C code, I have a build script that rebuilds a shared library 3-4 times for different CPUs (e.g. with and without -mavx2) and then uses dlopen() to choose the right one at runtime. The actual SIMD code has a bunch of #ifdefs to choose the appropriate implementation for primitive operations.

GCC has function multiversioning [0][1] (__attribute__((target("sse4.2")))) to enable choosing a function implementation based on CPU capabilities at runtime, but that is not suitable for the use case I describe. I want my "high level" code to be the same, but using the right implementation for "primitive" ops like dot product. Using multiversioning would completely kill the optimizer and cause excessive register spilling and other unwanted artifacts.

[0] https://gcc.gnu.org/wiki/FunctionMultiVersioning [1] https://gcc.gnu.org/onlinedocs/gcc-6.2.0/gcc/Function-Multiv...

> This has to be done at a granularity coarse enough that the function call overhead doesn't kill the performance. It has to be entire loops, for example, not just some primitive operation like a dot product.

Yes, You are right! I understood, that they wanted to add the branch to _every_ SIMD instruction. I concur with branching outside of loops.

Re Rust: I expect that one could use a macro around the function (or annotate it, if the macro system is good enough) that generates these different functions and branches to the right one.

It is the typical approach taken by libraries in C and C++ too (though you can usually hardwire to avoid paying the initial cost). Say, Eigen, OpenCL, fftw, ATLAS to name a few.

You do you.. but why wait on the benchmarks? its the only thing I care about when I look at this.. I'm willing to forgo semantics if its absurdly fast

If you don't care as much about usability and care only about speed, there's already stdsimd which allows you to invoke SIMD intrinsics directly.

This crate explicitly focuses on usability, and given the limited author time, prioritizes that.

The people behind stdsimd haven't started this thread praising their work as being absurdly fast. Extraordinary claims require extraordinary evidence. A baseless claim accompanied by excuses such as "no time for proofs, working on aesthetics" to avoid presenting evidence that would support those claims doesn't build any confidence on the original claim.

For the record, the author of `faster` did not post to HN. Secondly, since this is just a pleasant wrapper around stdsimd, I wouldn't expect performance to degrade severely. Finally, this is still a maturing project, and no where on their project page does the author claim that their work done.

lay off the unnecessary aggressiveness.

>The people behind stdsimd haven't started this thread praising their work as being absurdly fast.

Neither did the programmer of this lib.

>Extraordinary claims require extraordinary evidence.

Only if one's goal is to convince people who can't be bothered to do the check for themselves. Otherwise one can just make the claim, and let others (if they are interested) sort out whether it's true.

Reply on news.ycombinator.com