We use Pact[1], but the idea of flakeyness is interesting. Would you end up with non deterministic builds with the flakeyness? I imagine you could get enough 404's to trigger a Circuit Breaker in your application and this would propagate to tests sometimes failing.