For example, doing dangerous thing might be faster (no bound checks, weaker consistency guarantee, etc), but it clearly tend to be a reliability regression.
And how does performance improve reliability? Well, a more performant service is harder to overwhelm with a flood of requests.
Of course this is a very artificial and almost nonsensical example, but that is how you optimize bounds checks away - you just make it impossible for the bounds to be exceeded through means other than explicitly checking.
But also the stuff you tend to do to make it fast makes it more reliable.
Local caches reduce network traffic. Memory is more reliable than network IO so it improves reliability.
Reducing lookup calls to other services (e.g. by supplying context earlier in the dependency chain) makes it faster and more reliable.
Your code will probably branch less and become more predictable too.
And often the code is simpler (sometimes not when a performance hack is used)