one interesting difference is that memory prefetching/preloading has such a huge difference, whereas on arm/x86 it barely makes any difference (in my experience) that it's not even worth doing
also, doesn't allwinner d1-h have simd registers?
one interesting difference is that memory prefetching/preloading has such a huge difference, whereas on arm/x86 it barely makes any difference (in my experience) that it's not even worth doing
also, doesn't allwinner d1-h have simd registers?
i'm using an m1 laptop, so my sgemm implementation relies on arm neon vector extension, and i saw literally no performance improvements when i added prefetching, which could either be a result of their speculative optimisations or an issue in my own implementation
but i have noticed some ppl saying that adding prefetching on x86 has little performance improvement for sgemm, bc it's a scenario in which it's easier to predict the next memory address that's gonna be used
I hope that Apple will implement SVE/SME one day which will allow us to leverage this hardware directly...
meteorlake and zen4 (8000) also both have npus which are similar. https://www.pcmag.com/news/the-meteor-lake-npu-meet-intels-d...
On M3 a singe matrix block can achieve ~ 1TFLOP on DGEMM, I assume it will be closer to 4TFLOPS for SGEMM. The Max variants have two such blocks. Didn't do precise benchmarking myself, but switching Python/R matrix libraries to use Apple's BLAS result in 5-6x perf improvement on matrix heavy code for me.
Interesting. Do you have a reference or more information on this?
It should be pretty easy to find a MWE that shows the behavior. Something like summing an array of int* should do it.