pthread_join is a blocking call, and therefore against the spirit of lockless programing. pthread_join will block until the child-thread exists, which means there's a lock in there somewhere. I personally think the first example they show is a terrible example as a result.

------------

However, the smp_store_release and smp_load_acquire discussion is good... very good. But I'm not sure if there's enough discussion or theory to discuss why this is required.

The gist is: L1 caches on POWER9 or ARM are allowed to reorder loads/stores.

    thread 1                            
    --------------------------------  
    a.x = 1;
    message = &a;                    
                                      

"a.x = 1" should happens-before "message = &a". However, L1 caches do NOT always follow this logic. "message=&a" could happen first from the perspective of DDR4 RAM.

How? Why? Well, maybe message=&a is least-recently-used, while the thread continues to read/write to a.x variable. So later on, when the cache updates DDR4 RAM, the message=&a happens-before a.x=1 (from a DDR4 RAM perspective).

This never happens on x86 (the x86 Cores preserves all store orderings). But it is potentially possible on ARM or POWER9. (There are other possible reasons for reorderings: Out-of-order computations on the CPU, Load/store forwarding on the Core, as well as compiler optimizations).

It turns out that violating the total-store-ordering requirement allows for L1 caches and CPU-cores to operate slightly faster. Thanks to overly aggressive multicore processors and architectures (like ARM or POWER9), we have to deal with this non-obvious ordering violation.

---------

The barrier forces the happens-before relationship to be preserved. You need a barrier on two sides: the write side (release) and the read-side (acquire).

---------

Anyway... "happens-before" is about as bloody simple as you think it is. The issue is that "happens-before" is now violated on a regular basis on modern CPUs in multithreaded environments.

Forcing "happens-before" relationships to happen in the expected order is now a requirement for multithreaded programmers.

Note: all lock() and unlock() functions innately have the proper happens-before relationships in the right place. So a typical programmer who uses mutex_lock() will never have to worry about this happens-before reordering. All synchronization mechanisms: locks, semaphores, condition variables, etc. etc. need to be carefully written with these acquire / release barriers to work on modern architectures.

However, if you wish to step into the realm of high-performance lock-free algorithms, you must absolutely understand the modern processor, and how it violates happens-before relationships. (as well as understanding the memory barriers that RESTORE the happens-before relationship).