Intel AMX

Somewhat related, there is ongoing work on an integrated (in vector registers) matrix extension for RISC-V: https://github.com/riscv-admin/integrated-matrix-extension/b...

They are currently discussing the data layout, as the extension should be vector length agnostic, just like the vector extension:

    Option A: One matrix per vector registers.

    Option B: One matrix in multiple vector registers.

    Option C: Multiple matrices in one vector register.

    Option D: Streaming buffers for the input matrices.

    Option E: Variable matrix representation.

My favorite of the proposals so far, is to use the vector registers as the accumulator and add a small ammount of new state for the source operands (Option D): https://github.com/riscv-admin/vector/blob/main/minutes/2023...

full title is:

Accelerate Artificial Intelligence (AI) Workloads with Intel Advanced Matrix Extensions (Intel AMX)

and it's a brochure ("Solution brief")

I honestly feel cheated by intel with my 13th gen cpu. They took away avx512, even though the P cores are capable of it, but also did not ship AMX with it.

I'm honestly more interested in Apple's AMX "co-processor", in my tests it often beats the GPU in ML tasks with smaller batch sizes. It's really cool. I hope it becomes more common for training/inference/numerics capable accelerators to be included in consumer hardware.

Apple's AMX is really under-documented, while the instructions were reverse engineered, Virtually no benchmarks are available comparing current chip generations, models and variants.

https://github.com/corsix/amx https://github.com/philipturner/amx-benchmarks

Curious to see how Intel's AMX stacks up to Apple's AMX in real world perf.

It just compares to itself and AMD. It is missing the comparison to GPUs