> that's frustrating for everyone behind them in the best case at best... and I imagine possibly even more dangerous in the worst case, no?
No.
Waymo put the myth to bed [1]. Even if you might piss off a speeder, driving the limit in speeding flow remains safer as the handling advantage (frequency) and exponentially-lower energy in the event of a collision (magnitude) dwarf other effects.