The x86 Advanced Matrix Extension
fuse.wikichip.org
fuse.wikichip.org
It is an awful lot of registers for a feature that few programs may use. There's the risk that bfloat16 is a fad and five years from now it is used hardly at all. At best it is for a full stack perception-and-synthesis feature about as good as
https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video
Most of all it drives me nuts that Intel is going down the fixed-length SIMD instruction route (going wider) where the instructions you write are specific to the structures of the processor you run on. Machines like the ILLIAC, Cray, and the vector processor for the 3090 mainframe would automatically chunk the work so that you didn't need to rewrite your code for a future wider model.
ARM is doing it the right way:
https://alastairreid.github.io/papers/sve-ieee-micro-2017.pd...
But as for architectural extensions, there clearly are customers asking for these things, even TSX which has never worked.
I was just wondering what you could do with 8KiB of register memory, if you ignored the intent of the registers outright (assuming you can load/store to the classical registers from these registers).
Although x86's SIMD extensions have a bunch of crippling issues, fixed-width instructions are fundamentally more general because they work both for vectorizable length-N loops, and for the many, many cases where you have a fixed units of work that you can perform in parallel. If you design your architecture to depend on loops to extract full performance, you cut yourself off from many productive uses of vector instructions outside of loops, and make it more difficult to juggle cases where you have more than one loop dimension.
Upgrading to new architectures with longer supported vector lengths is best left to recompilation and variable-width vector libraries.
Any win all these features bring can also be achieved with ARM or Zen by adding more cores, with the exception of those few cases where you have huge computational tasks that cannot be efficiently parallelized and where there are few discrete jobs to allow for coarse grained (job/task) parallelization. There are not many of these.
Meanwhile all these features are going to make Intel chips even more complex, making bugs more likely and making iteration more costly.
My read is that Intel is shooting for maximum possible single threaded performance because they can't compete on power efficiency or many-core. They can't compete in those areas because their process nodes are not as small as what TSMC can offer, and both (most) ARM chips and AMD are using TSMC and fabricating at 7nm and soon 5nm. (Yes I know nm node sizes are no longer directly comparable, but they are ahead of Intel and likely to stay ahead unless Intel can really push hard on fab engineering.)
The good thing is that, as any FP number, can represent a very large range of values, so I'd expect percussion and other things with very high-frequency transients to sound nice. My bet is it'll sound "colorful". IEEE float16 should sound better.
But I'm no expert and I don't have the time to write something that force a good high dynamic range track to be rounded to the nearest bfloat16 and then expanded back. Also, I don't have gear good enough to hear anything better than CD-grade audio.
- The -66dB noise floor is pretty bad, and it accumulates at every step.
- 11 bits is probably not enough for filter coefficients. So your filters would likely be running with single precision floats, at least. Even low-cost DSP chips tend to give you a big chunk of bits for your filters.
- Naïve oscillator designs would accumulate a lot of error. Back of the envelope calculation, if you wanted an oscillator at C4, you’d likely be around 1/4 tone sharp or flat unless you ran the oscillator at higher precision.
I’m definitely of the mind that bit depth is overrated in music. 16 bits is great-for mastered music and simple tweaks. By contrast, from my experiences writing DSP code, it often makes your code simpler and faster to run at higher depths and sample rates, and then convert to e.g. 16 bit as the very last step. The problem is that squeezing good output from low precision or low sample rates requires more complicated and slower algorithms.
The other extensions do not trigger it, not even AVX256.
With AVX512 is not always a win, and you don't even know until you try in particular hardware.
When that’s the case, especially if the numbers being crunched are 32-bit floats, there’s not much point of doing it on CPU at all, GPGPUs are way more efficient for such tasks.
However, imagine sparse matrix * dense vector multiplication. If you rarely have more than 4 consecutive non-zero elements in rows of the input matrix, and large gaps between non-zero elements, moving from SSE to AVX or AVX512 will decrease the performance, you’ll be just wasting electricity multiplying by zeros.
This is also ignoring the fact that none of these penalties come into play if you use the AVX512 instructions with 256-bit or 128-bit vectors. (This still has significant benefits due to the much nicer set of shuffles, dedicated mask registers, etc.)
For general guidelines on when to use AVX-512, this (older post) remains best guide I've seen: https://lemire.me/blog/2018/09/07/avx-512-when-and-how-to-us...
Among other things, it depends on the workload and the exact processor. You can find plenty of cases where AVX512 makes things faster. You can also find cases where the entire system slows down because it is running sections of AVX512 code here and there—apparently, for certain Intel processors, the processor needs to turn on top 256 bits of the register files and interconnects, and to get full speed for AVX512 it will alter the processor’s voltage and clock speed. This reduces the speed of other instructions and even other cores on the same die (which may be surprising).
While the specifics may be new, the generalities seem familiar—it has long been true that a well-intentioned improvement to a small section of your code base can improve performance locally while degrading overall system performance. The code that you’re working on occupies a smaller and smaller slice of your performance metrics, and meanwhile, the whole system is slowing down. There are so many reasons that this can happen, dynamic frequency scaling with AVX512 is just one more reason.
Agner’s microarchitecture.pdf says about Ryzen “There is no penalty for mixing AVX and non-AVX vector instructions on this processor.”
Not sure if it applies to Zen 2 but I’ve been using one for a year for my work, AVX 1 & 2 included, I think I would have noticed.
Just because they are split doesn’t mean they run sequentially. Zen 1 can handle up to 4 floating point microops/cycle, and there’re 4 floating-point execution units, 128-bit wide / each (that’s excluding load/store, these 4 EUs only compute).
Native 256 bit are even faster due to less micro-ops and potentially more in-flight instructions, but I’m pretty sure even on Zen 1 AVX is faster than SSE.
However, when instruction decode/retire is the bottleneck, AVX can be faster. I remembered this can be the case on Intel Sandy Bridge (first-gen AVX, double pumped, retire 3 instructions/cycle), where AVX can sometimes be faster (usually it's not that different)
With recent CPUs from both Intel/AMD able to at decode/retire at least 4 instructions per cycle this really cease to be the case.
Yes. Another possible reason for that is instructions without SSE equivalents. I remember working on some software where AVX2 broadcast load instruction helped substantially.
Presumably there's a character limit on HN titles, and '/' isn't allowed either?
It's sort of amazing the amount of performance you can squeeze when you fix your OS and cpu architecture.
There are so many extensions: https://software.intel.com/sites/landingpage/IntrinsicsGuide -- are we suppose to write our own libraries to leverage these or do we need to file tickets with our favorite compilers for them to develop these optimizations?
Oh, one more questions, how do these overlap with AMD?
AMD usually implements them but with a couple of years of delay.
For example AVX512 is not implemented, and we had to wait for Ryzen 3 to have the same AVX capabilities as Intel (2 AVX units per core instead of one).
Intel deploys their own GPU some time in the future.
Interestingly, there does seem to be a new extension (see §3.2.6 of https://software.intel.com/content/www/us/en/develop/downloa...) that, on first glance, looks to be a per-thread enable/disable bit for using these registers, which suggests that an OS could make a process-level capability to enable/disable AMX and thereby not bother saving these registers on context switches if it's switching to a process without AMX.
I'm looking forward to RISC-V's V extension, which is due to become standard around September. Unlike AVX512 and friends, this one is vector size agnostic.