That said, if I had to work with awful code and I wasn't allowed to touch it or make it better, I would definitely leave in a heartbeat.
But when the stakes are higher, the risk/reward ratio just isn't there. You can refactor the code to make it more sensible or pleasing to your eye. Two problems with this: 1) Someone else may disagree and refactor it again. 2) When it blows up or has unintended consequences, and you are now on the hot seat, you find yourself working on Saturday asking yourself if it was worth it at all. Thus the fear sets in.
I think it's also implied that, in a situation where technical debt isn't appreciated, it's unlikely that resources would be given to pay it down. Instead, it sits there making all work more frustrating, whereas a shop with minimal technical debt spends more time creating new, more robust, code.