I would add this awesome talk which never got much traction (outside of comments in other threads).
Testing Distributed Systems with Deterministic Simulation [video] - https://news.ycombinator.com/item?id=8345030 - Sept 20, 2014 (6 comments)
Testing Distributed Systems with Deterministic Simulation [video] - https://news.ycombinator.com/item?id=8345030 - Sept 20, 2014 (6 comments)
I've added that thread and also https://news.ycombinator.com/item?id=23289593 to the list above.
I just can't imagine tools like this not winning out on a long enough timeframe, particularly with compiler support.