Sounds like the chaos monkey program Netflix used to test is infrastructure by introducing random amounts of latency, switching off certain parts of the infrastructure etc