another reason to not refactor is management will start out supportive of it but some weeks in they will wonder wtf their engineers are doing all day and why aren't they building the new pile of stuff they want built yesterday.
The worst hostility is often from management that used to be technical but never learned how to refactor effectively so they have what they think is justifiable scepticism. s/refactoring/pairing or any other technique that is good in theory but requires skill to implement.
I usually find that people without technical background are willing to follow the advice of the developers that work with them, taking the things that work with them into their next roles. If refactoring worked in the beginning they'll be open to it in future, if it didn't work they'll dismiss it.
Edit: also - what everyone else said - "refactoring" isn't a business goal, it's one part of achieving the goal. Achieve the goal and no-one cares how you did it. ("add a new button to that widget" isn't a business goal either so there is a tricky aspect of getting management to understand that they should be bringing you problems/goals not solutions)
Like rather then refactoring the entire application to support a faster cache or something you can just add in another microservice that handles caching of the data.
If some way you write the services is poor you can rewrite one of the services a new way and easily evaluate if it's actually a good idea to change the rest or not.