Hacker News Reader

Comment by vegetablepotpie | Hacker News Reader

Apple added custom instructions on top of the ARM instruction set to do matrix operations.

Matrix operations are used a lot in some algorithms, such as in computer graphics and machine learning and these instructions help those operations go faster.

Although the ARM ISA has a multiply and accumulate instruction, it would take many more instructions on a standard ARM core to compute a matrix with only standard instructions. That enables the M1 chip to be faster in this case when developers use these instructions in their applications.

View on HN

Correction:

^when applications use these instructions; either as a result of use of a system API/framework, of the compiler optimizing code into these M1 instructions, or of the developer writing M1 assembler themselves.

It is very likely that the majority of cases that use these instructions will be compiled by Xcode without the knowledge of developers that these instructions exist at all.

Apple might use these instruction in CoreML/Accelerate Framework and not integrated to LLVM as the author said:

> This is an undocumented arm64 ISA extension present on the Apple M1. These instructions have been reversed from Accelerate (vImage, libBLAS, libBNNS, libvDSP and libLAPACK all use them),

So we still cannot understand all of them from LLVM source code.

To highlight where we're disconnected, my phrase:

"cases that use these instructions will be compiled by Xcode"

does not imply that Xcode comes equipped to translate user-provided code into these instructions; however, by use of Apple frameworks, Xcode could still regardless output binaries that use these instructions, without needing to provide us any manner to do so ourselves.

I suspect most people aren't compiling framework-provided binary code into their binaries, using static or whatever it's called these days and so I believe those usages will end up reserved for the frameworks — but since someone is certain to try and consider 'the binary output of Xcode' to include any frameworks referenced by it if it supports their point, I'm taking care to cover all possible cases where output of those instructions could occur without a translation phase using LLVM or etc.

Thanks for your explanation, I didn't take this into account

No worries, I’ll think about how to do better with my wording next time too.

It's interesting, in as much as there's been a constant bickering around whether the GPL vs Apache license for GCC and llvm means anything; to date, there general view has been that there's no evidence of anyone using LLVM to build compilers with closed extensions that are not released to the community; this is perhaps the first case of that.

Is there any evidence that Apple has an LLVM compiler with these extensions, versus using asm blocks within the Accelerate library source?

There's certainly prior cases of Apple shipping branches of LLVM (most notably adding arm64 support, which took about half a year to merge in upstream), but the mere existence of these instructions is little evidence of any branch?

I have heard of none, though I do not speak for others.

No idea what the story is with AMX instructions, but Apple has been shipping an arm64e compiler for quite a while with changes that have not been upstreamed.

Similar to Altivec from the PowerPC days?

Altivec is regular SIMD like NEON or SSE/AVX

> That enables the M1 chip to be faster in this case when developers use these instructions in their applications.

Or more typically the compiler will emit those instructions and everyone gets them for free.

only if the compiler knows they exist

That's actually generally not true WRT SIMD/vector instructions (see eg http://yosefk.com/blog/humans-and-compilers-need-each-other-... which contrasts this with the more compiler-friendly VLIW case); they impose nontrivial requirements on data structures and ABI, so the application code needs to at least be aware of the possibility of vectorization, even if it's agnostic to whether the implementation actually uses it.

Reply on news.ycombinator.com