RISC-V and Floating-Point
fprox.substack.com
fprox.substack.com
What Iʼm slightly confused for is that all these extensions, useful for a minor part of applications, arenʼt moved to longer instructions (6-byte).
Also groups of extensions are consolidated into Profiles, so in practice you don't really care about individual extensions. You'll only care that the hardware supports eg RVA23.
If you're building your own embedded hardware then you determine what your needs actually are e.g. do you need double precision? half precision? vector?. Then you choose a chip implementing that. Then you copy the ISA string from your chip's documentation to the `-march=` argument for GCC/Clang and be happy.
It's not hard and you don't have to think about it unless you very specifically want to.
That process doesn't depend on whether it's an x86 or an Arm or a RISC-V.
That's why x86 people refer to the HUGE document maintained by Agner Fog.
If you want your code to run well on all standards-compliant implementations then you write according to the ISA documentation, in this case RVA23. Or ARMv9-A. Or x86_64 v3.
Riscv is basically repeating the same mistake X11 did. A minimal base that could be varied endlessly by combining extensions. I didn't work for X11. Some extensions became de facto mandatory (shm) while others fell by the wayside. But you could never rely on the availability of a given extension because someone somewhere might not have had it or disabled it. Then Wayland came along and cleaned up the gazillion extensions mis-design because it was a huge PITA. Riscv will get there too, sooner or later.
If you are on that level then you know pretty well what you are targeting. And even then in 99% of cases you just look at the top level profile.
If you do performance analysis for some specific embeded project that is not using a standard profile, then its a bit more work, but hardly impossible.
It's here to stay, coming to a device near you Real Soon Now (tm).
I have in my hands one of the new SpacemiT K3 machines. It arrived today. I'm comparing it to several other things, and finding that it is pretty comparable to a "late 2012" Mac Mini with a i7-3720QM with base 2.6 GHz turbo 3.6 GHz running Ubuntu 24.04. They are quite close in feel for general use, web browsing, code editing, watching YouTube etc. The Mac is a little faster on many things, a LOT slower on others (anything that can use 8 cores, obviously).
You might say that's not "high performance" but we thought it was pretty good a dozen years ago.
The previous SpacemiT K1 chip two years ago was more like one of the last Pentium IIIs or PowerpC G4s, except with a lot more cores.
SpacemiT have a next generation K5 coming out, they say, at the end of the year. Tenstorrent have their new Ascalon-X core comparable to Apple's late 2020 M1 — and designed by the same guy who designed the M1. They've taped out a chip using that and say they'll be selling a dev board in Q2 or Q3. For now the first version is using an old chip process and it will be running at half the clock speed of the M1, but that's still going to be a very decent machine.
The HiFive Unleashed was of course 8 years ago. Since then there have been the HiFive Unmatched (roughly like Cortex A55) and the HiFive Premier P550 (a bit better than Cortex A72, other than no SIMD).
Definitely sounds pretty high-performance compared to basically every RISC I've seen (and including nearly every cell phone I've ever owned with the exception of the Apple ones).
Tenstorrent is awesome, can't wait to see if I can afford any of their hardware in 5ish years. I miss when you could buy TPUs as a consumer (Coral etc.)
- https://milkv.io/jupiter2-dev-kit
More here: https://redd.it/1tcrx4o
Of course, neither of these are machines. And the average bystander probably isn't used to importing computer parts directly from China, either.
https://x.com/BruceHoult/status/2056911834737975635/photo/1
I've already posted on github my first project written on and for it today:
https://github.com/brucehoult/k3_ai
Sipeed have posted photos four days ago of the first batch of customer orders going out:
https://x.com/SipeedIO/status/2055549071931404291
> the average bystander probably isn't used to importing computer parts directly from China, either.
It won't take long for them to be available on amazon, just as D1 and JH7110 and K1 boards are now. e.g.
https://www.amazon.com/Orange-Pi-RV-Frequency-Development/dp...
Hopefully, one of these systems gets produced in such large quantities that there's some pressure to add mainline Linux support.
https://store.deepcomputing.io/products/dc-roma-risc-v-mainb...
It's the same for other ISAs.
> What Iʼm slightly confused for is that all these extensions, useful for a minor part of applications, arenʼt moved to longer instructions (6-byte).
Because these instructions don't need it. There will be future >4-byte instructions, for things thay can't resonably be done in 4-bytes, e.g. much larger immediates.
Also fun: FEAT_LittleEnd, FEAT_MixedEnd, FEAT_BigEnd
All of that was just 64-bit ARMv8.x-a, there is a lot more stuff, once you go to R or M profiles, 32-bit and previous versions.
The reason this is mostly not a problem, is that distros converged on a minimum of 64-bit ARMv8-A + FP + SIMD, which will also happen with RVA23 for RISC-V.
Just for fun, here are the Zen4 ISA flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3 dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx512_bf16 clzero xsaveerptr arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload avx512vbmi umip avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid fsrm
Compared to RVA23 written out: rv64imafdcbv_zicsr_zicntr_zihpm_ziccif_ziccrse_ziccamoa_zicclsm_zic64b_za64rs_zihintpause_zba_zbb_zbs_zicbom_zicbop_zicboz_zfhmin_zkt_zvfhmin_zvbb_zvkt_zihintntl_zicond_zimop_zcmop_zcb_zfa_zawrs_svbare_svade_ssccptr_sstvecd_sstvala_sscounterenw_svpbmt_svinval_svnapot_sstc_sscofpmf_ssnpm_ssu64xl_sha_supm_zifencei
M, A, F, D, Q, C, B, H, Zicsr, Zifencei, Zicntr, Zihpm, Zihintpause, Zihintntl, Zicbom, Zicbop, Zicboz, Zicond, Zicfilp, Zicfiss, Zimop, Zca, Zcb, Zcd, Zce, Zcf, Zcmp, Zcmt, Zcmop, Zclsd, Zilsd, Zmmul, Zfh, Zfhmin, Zfa, Zfbfmin, Zfinx, Zdinx, Zhinx, Zhinxmin, Zaamo, Zalrsc, Zawrs, Zacas, Zabha, Zalasr, Zba, Zbb, Zbc, Zbs, Ztso, Zbkb, Zbkc, Zbkx, Zknd, Zkne, Zknh, Zksed, Zksh, Zkn, Zks, Zkt, Zk, Zkr, Zve32x, Zve32f, Zve64x, Zve64f, Zve64d, Zve, Zvl32b, Zvl64b, Zvl128b, Zvl256b, Zvl512b, Zvl1024b, Zvl, Zv, Zvfh, Zvfhmin, Zvfbfmin, Zvfbfwma, Zvbb, Zvbc, Zvkb, Zvkg, Zvkn, Zvknc, Zvkned, Zvkng, Zvknha, Zvknhb, Zvks, Zvksc, Zvksed, Zvksg, Zvksh, Zvkt, Sm1p11, Sm1p12, Sm1p13, Smaia, Smepmp, Smstateen, Smcdeleg, Smcsrind, Smcntrpmf, Smrnmi, Smdbltrp, Smmpm, Smnpm, Smctr, Ss1p11, Ss1p12, Ss1p13, Ssaia, Ssccfg, Sscsrind, Sscofpmf, Sstc, Ssqosid, Ssdbltrp, Ssnpm, Sspm, Ssctr, Supm, Sv32, Sv39, Sv48, Sv57, Svinval, Svnapot, Svpbmt, Svadu, Svvptc, Svrsw60t59b, Sdext, Sdtrig
That doesn't look very short to me.
These are grouped into profiles, like "Skylake" or "Cortex-M33" or "Neoverse-N1." The main issue for RISC-V isn't the number of instruction set extensions, it's the number of profiles. RVA23 is one single blessed profile, but many chips will add a few more instructions or include fewer than RVA23 based on age of the chip.
Beyond that, what other extensions a particular board or chip supports, doesn't affect regular uses like web browsing. Specific apps or software libraries may use an ISA extension if present. Same as for other ISAs.
Code for embedded systems is optimized for the exact cpu in there. Same thing for highly specialized jobs (scientific / datacenter type stuff).
In short: yes, fragmentation wrt ISA extensions, hardware & software support exists. In practice, it isn't a big problem as some claim it to be.
Basically, outside of MPX, and the confusing lineage of AVX-512 on client versus server parts, x86 is pretty strictly additive.
The only reason they're optional is because I'm using the same instruction set on my Pico, so no it doesn't have floating point, and I believe it has integer divide but I wouldn't be surprised if it didn't.
And the extensions are in groups, a good chunk of which are compressed instructions, which unless you're writing assembly, you don't need to worry about.
In fact most of this you don't need to worry about unless youre writing assembly.
What? There are individual chips with nearly that many ARM variants, including incompatible ISAs (M0 vs R52) and compatible-but-very-different-performance-characteristics implementations of the same ISA (M4 vs M7, say). Even figuring out what portion of code can be shared across which cores (and for those that distinguish between ARM and Thumb mode, what mode that code can be called in), vs what code needs duplicate versions for different cores for correctness, vs what code needs duplicate versions for performance but not correctness (which changes as the code usage pattern evolves) can be a challenge on a single chip; I can't imagine a world where you can think about only five across an entire industry.
No they are not. See the Intel Software Programmer Volumes. Highly detailed, highly structured and highly specific.
Tired finding similar quality documentation on ARM and RISC-V and came empty handed.
They are absolutely in denial as to the downstream effects of this on the software ecosystem. Android, for example, for native support had enough fun dealing with relatively few ARM variants (and x86/MIPS etc), and identifying chip features at run time was reliant on the board support software getting it right (hint: it didn't).
I would have loved if the article mentioned the efforts at integrating Posits [0] in risc-v. While IEEE734 compatibility will obviously be necessary for any foreseeable future, it would be nice if the industry could settle on a better alternative which avoids many of the flaws with IEEE floats.
One of the issues with posits from a numerical perspective is that it's main claim to fame of being more accurate is handled by having the number of precision bits being dependent on the exponent value, which means it's a return to the days of having to scale your input so that the values are around unity to maximize, rather than the promise of floating-point of being scale-independent.
Care to elaborate? All the analyses I've seen show that posits achieve generally higher precision at the same number of bits compared to IEEE floats. You don't even necessarily need to care about scaling your inputs to be close to ±1; most problem domains already naturally have this!
The "quire" accumulator also enables new kinds of algorithms that are just not possible with IEEE (technically one can have a fixed-point accumulator for IEEE floats, but it's not as practical).
... by repeating lots of the flaws that led to IEEE754, requiring extra accumulators (the "quire") and hardware to do basic ops since the posit format alone fails, and making numerical analysis a complete mess, breaking the ability to write correct numerical algorithms.
They lose precision over large dynamic ranges, making algorithms fail on many inputs, without extreme care (and loss of accuracy over such ranges), lack of NaN/inf makes them fail on lots of other issues (and there are algorithms requiring NaN and inf behavior under IEEE754 for performance - I'll list one I recently made below...), this lack makes it harder to debug where algorithms broke, costing development time, ....
The algo I recently developed needed to find extrema of cubics over a finite range. This requires solving a quadratic. A quadratic root solver can have /0 = inf and sqrt(-) = NaN cases, which are often fiddled with using branches.
In my case I knew I'd be doing these in batches, and wanted C/C++ code to auto vectorize and do them in SIMD, and did not want to pay the cost for branches. This speed up the flow by about 8x on almost all larger processors, at the cost of some slots having NaN or inf. Those with NaN or inf had underlying cubics I could discard. So by using the IEEE754 aware multi parallel root finder (written in strd c++), I could check that the roots were in my interval (also parallelized) as a <= root <= b, which fails for root being NaN or inf. This check is also parallelzied.
All in standard C++, no hint of parallelization intrinsics, handled by modern compilers perfectly, and getting massive speed gains.
This is but one place NaN and inf are extremely useful. This type of use appears all over in scientific computing, graphics, pysics sims, etc.
Posits cannot handle this type of stuff.
> lack of NaN/inf makes them fail on lots of other issues
You comment repeats several times assertions of the form "fails on lots of stuff", but does not really elaborate further x) But that aside.
Posits do have a NaN value, with the difference being that only 1 bit pattern is reserved to it instead of potentially quadrillions of them. This makes a huge difference especially at lower precisions, where NaN bit patterns waste an appreciable amount of the total set of values. They do not have inf, they simply do not overflow.
> In my case I knew I'd be doing these in batches, and wanted C/C++ code to auto vectorize and do them in SIMD, and did not want to pay the cost for branches. This speed up the flow by about 8x on almost all larger processors, at the cost of some slots having NaN or inf. Those with NaN or inf had underlying cubics I could discard.
Can you point to your code? I'd be very very surprised if it somehow cannot be rendered using posits.
Pathological cases do exist, of course. Nobody claims that posits are uniformly better than IEEE floats, such a claim being obviously, trivially nonsensical.
> All in standard C++, no hint of parallelization intrinsics, handled by modern compilers perfectly, and getting massive speed gains
Speed gains over what? x) I don't get it. Over branchy or scalar IEEE code? If yes, what's the relevance of that?
And yes, a 40-year old format with decades of hardware and compiler support works better in practice than any conceivable alternative... because all such alternatives are proof-of-concepts with no hardware or compiler support! There's no commercial hardware of any kind for posits, let alone SIMD. I really fail to see your point here.
I really want to see hardware built around posits. This is not because they're necessarily superior to floats (they aren't in all use cases), but just because we need some diversity. Too much standardization is bad for innovation, and not everything that's settled should remain that way.
https://www.sigarch.org/posit-a-potential-replacement-for-ie...
see f.e.: https://wren.wtf/shower-thoughts/marks-magic-multiply/