From this point of view, I guess you should make an exception and treat tech debt complaints more seriously when it is coming from the devs who wrote the original code.

Sometimes in order to meet business needs and get things out the door you have to cut corners that you need to go back and take care of later, or you are crippling the future growth and stability of the project. I like your final point "if every change to a bit of code introduces new bugs then that might be a reason to tidy it up".

I think sometimes proper testing is the first thing to go in a time crunch, and good tests improve velocity. If you know that the entire app is being tested automatically, you can develop a lot faster and more fearlessly. Plus writing good tests inherently makes the original code better. You find and fix hidden bugs, and you refactor the code to be more testable which makes it better in other ways.