I didn't really get the example, futex is itself a syscall, so how does implementing mutex in terms of futex help you create a "fast userspace" mutex? The example presented looks to do almost nothing in userspace. Am I missing something?
mutex_lock makes 0 system calls in the fast case (when cmpxchg succeeds on the first try). mutex_unlock makes 1 system call always, but TFA mentions that it can be optimized by adding some user-space complexity to check if signalling might be required (if you can demonstrate zero waiters, then the FUTEX_WAKE is not needed).
Yes, usually you keep a (saturating) waiter count, at the limit just one additional bit.
The example really should have fleshed this out, because most of the point of the feature is that you can have a pathway that involves no kernel operations in the common case. The rest of its point is that if you do end up waiting, the kernel scheduler knows about it.
when the cas (compare and swap) succeeds in user space, the subsequent futex call to wait will not be made, i suppose.