When Ldaxr Doesn't Work: Exclusive Accesses and Cacheability on AArch64

Ldaxr works on any Aarch64 CPUs, but on Armv8.2-A and later CPUs, i.e. Cortex-A75, Cortex-A55 and all later CPUs, it is better to not use the non-deterministic ldaxr but to implement mutual exclusion and dynamic partitioning of shared resources using the fetch-and-add (ARM mnemonic LDADD) and fetch-and-store (a.k.a. atomic exchange a.k.a. atomic swap) (ARM mnemonic SWP) instructions, which can provide deterministic and fair behavior.

Also for shared bitmaps, which are useful in many algorithms, bits can be set and reset atomically with fetch-and-and-with-complement (ARM mnemonic LDCLR) or fetch-and-or (ARM mnemonic LDSET) instructions.

While for mutual exclusion or dynamic partitioning ldaxr is always a bad choice, in theory there are some algorithms for optimistic access to shared resources (a.k.a. lock-free) that can benefit from ldaxr, but even for optimistic access the most useful case is that of optimistic read-only access, which does not need ldaxr.

The fact that the first version of Aarch64 did not have anything else for implementing atomic accesses except ldaxr, was its biggest mistake, which was corrected quickly in Armv8.1-A and the CPUs with an ISA older than Armv8.2-A, i.e. Cortex-A72 and Cortex-A53, have been obsolete for almost a decade (though annoyingly one can find even today some embedded computers with such archaic CPU cores, which force the use of inefficient algorithms in multithreaded applications).

There is quite a lot of research work in formally proving correctness of such primitives in relaxed-order memory architectures. You might want to take a look in some of them:

https://github.com/open-s4c/vsyncer

https://github.com/hernanponcedeleon/Dat3M

https://github.com/MPI-SWS/genmc

When cache is disabled, then typically exclusive ops are passed as is direct to the interconnect the core is plugged into. In this case it appears that the SOC doesn't implement an exclusive monitor, but did at least remember to generate aborts for these operations

Very smart debugging step to switch to standard load and store. This is often missed. A bit of forest for the trees.