I'd heard over the years that working with isochronous systems was difficult. I'd done a number of real-time systems before, and written OS schedulers and NTP-like systems and so forth. A little audio work should be a walk in the park, right? A little manly-man programming from the wrist and we move on to real problems. So I walked into an audio project thinking that "Oh, this latency and synchronization stuff, how bad could it really be?"

Bad.

Nailing down buffer-bloat and sources of latency and jitter in an isochronous system took several months, the last few weeks of which were 80-100 hour weeks just before ship. Several times we thought we'd fixed the issues, only to find that our tests had been inadequate, or that some new component broke what we had built.

I remember nearly being in tears when I finally realized what the clock root of the audio system actually was, and that it wasn't what people had been using. From there everything fell into place.

Don't pull the number of buffers you have out of thin air ("Oh, six is enough, maybe twelve, don't want to run out in my layer of the system, after all...") Don't assume your local underflow or overflow recovery strategy actually works in the whole system. Don't assume your underlying "real time" hypervisor won't screw you by halting the whole god damned system for 20ms while it mucks around with TLBs and physical pages and then resumes you saying, "Have fun putting all the pieces of your pipeline back together, toodle-oo!" Put debug taps and light-weight logging everywhere and attach them to tests. And know what your clock root is, or you will be sunk without a trace.

Isoch is hard.