Take a simple 4-wide dot product for example, on x86_64 you'd have 3-4 different implementations (SSE2, SSE3 w/ hadd, SSE4.2 w/ dpps). But the function itself is just a few clock cycles, and calling it via function pointer will eliminate any gains and you might as well compute it with a scalar loop at that point.
This is further compounded by inhibiting compiler optimizations. You can't use the dot product function in higher level code expecting it to be inlined and further optimized (which is really the key to performance) if it's behind a dynamic dispatch.
A sufficiently smart compiler could maybe propagate the dynamic dispatch above, so that all the dot products get inlined but all code using dot product would get emitted multiple times with different dot product implementations, with the dynamic dispatch only at the top level. This has a slight risk of combinatorial explosion, but there really aren't that many combinations of supported ISAs in real hardware out there.
Another option you can use without any special compiler support is to take all your performance sensitive parts and pack them into a shared object/dll, compile multiple versions with different compiler options and choose the correct dll at runtime. Or even build the entire executable a few times and have some kind of launcher pick the correct one.
I understand the desire for magic-indirection ergonomics; I just don't think the tradeoffs work out the same for code vs data.
It's a tiny bit harder to force an indirect call if that's desired for some reason; I think you'd need to write a slightly longer never inlined helper function to strip the constness from the pointer. It's doable though, just not directly provided by the language.
I appreciate that this doesn't fit Zig's call syntax, which is unlikely to change, so the long-term best case is probably some LSP marking based on the callee type.
1. IMO it'd be a bit more ziggish to branch at compile time. Zig has cross-compilation as a first-class feature, and you generally know the architecture you're targeting.
2. Whether you're branching at runtime or compile time, it'd be easy to build a vectored app with that behavior. The first thing that comes to mind is having code that's generic on the vector type (or bit width) and then just choosing which generic to instantiate in an inline for loop in your app's entrypoint.
3. A lot of the time you don't need that behavior. Select a vector width 8x too big, ensure your data is chunked into multiples of that, and rely on the compiler to break that down into a few instructions of the appropriate length. You can't effectively target a GPU that way, and it's not the same as hand-tuned assembly, but you get decent results on a vast array of problems.
I’ve seen people just compile for the lowest common denominator. E.g. faiss-cpu on PyPI doesn’t even include AVX2 and is slow as heck.
- You pay for it in build times, more IR, more optimizations.
- a lot of software is for servers, in that case you mostly know the arch before-hand. Multi-versioning is best for consumer software.
- The lowest spec computers, in consumer software, need the optimizations the most. And they don't have latest instructions.
- SIMD has biggest gains from memory optimizations, and those do not necessarily require the latest and greatest instructions.
But, this functionality is absolutely critical. It doesn't even have to be automatic. Just the ability to compile functions with certain ISA extensions enabled, and then only call them when the requisite CPU features are enabled is enough.
In a nutshell: https://github.com/BurntSushi/memchr/blob/8037d11b4357b0f07b...
const std = @import("std");
fn f(comptime width: comptime_int, value: i32) i32 {
const v = @splat(width, @as(i32, value));
return @reduce(.Add, v);
}
pub fn main() !void {
std.debug.print("1={d}\n", .{f(1, 5)});
std.debug.print("2={d}\n", .{f(2, 5)});
std.debug.print("4={d}\n", .{f(4, 5)});
}
1=5
2=10
4=20