- Huh, getting USB disconnects randomly, need to fix this
- Hmm, device driver seems to be fine
- Hmm usbnet driver seems fine
- Hmm xhci driver seem fine
- $5000 and one USB 3.0 protocol analyzer later...
- What the heck is this host controller doing?
At this point it was looking like a hardware problem but seeing as I didn't have the schematics or verilog of the host controller I could go no further (nor did I want too... I never imagined USB could be such a deep rabbit hole).
Seems like "Holy grail" bugs like this happen to need a thorough understanding of complex systems (which I usually like to hide under layers of abstraction).
For example, a bug in the implementation of a virtualized device looks like a hardware bug to the client and like a software bug to the host.
https://emulation.gametechwiki.com/index.php/Emulation_Accur...