Refactoring is low level work that is part of every task, it should not even be mentioned.
When you ask your mechanic to change wheels, you don't expect him to explain that he will have to take off the previous wheel and change the screws, that will be an item on the bill consuming half of the budget.
If I'm doing a bugfix task on a legacy site, I might note that it would be good to spend ~4h on a refactor, when the bug may have taken 0.25h to fix. In that scenario the benefits for refactoring are low, so we don't do it.
Alternatively, if I'm doing a 6h feature, and understanding and working with the bad code would take 6h, versus spending 4-6h on a wholesale rewrite, of course I would just slip the refactor inside the original task.
You have to keep your expectations realistic and your project managers informed. If your refactor suddenly blows out in time because you didn't fully comprehend your problem space, then you both haven't completed the feature or achieved the refactor. Maybe in your business a day extra is fine, in some businesses running over by a couple of hours can mean the ROI doesn't make sense on that task anymore.
The key is to be sensible and pragmatic. There is no one solution to all thinking. If there's anything any of these methodologies are trying to do, it's provide a framework for quickly communicating and reacting to the nuance of software development.
And that's why refactoring has a terrible reputation and developers can't be trusted. Now you're talking about replacing key components and altering the business logic under the pretext of refactoring.
This should NOT be called refactoring, this should be called replacing key components and revisiting the business logic.
Of course, management doesn't want that to be done without justification.
It sounds like you're not getting good communication from your developers or you're misunderstanding them when they say they need to refactor. Refactor is still the right word for it, but you need to find a way to communicate the extent of the refactor.
Cleaning up code as you go should be a habit I agree. It's slightly trickier when code reaches a "boiling frog" point and you need to take a look at the bigger picture and perhaps re-architecture. It often easy to keep chugging away at the tree's and not seeing the forest.
The downside with the scope of those kind of structural refactoring is that they touch a lot of code, so they're perceived as "risky".
Also developers get attached to the architecture they have learnt and know, and don't want to re-learn, even if the simplified code would be easier to learn for someone new
Sometimes you get a small change request that suggests a broader refactoring. You're pretty sure it will be valuable in future, but it would be a lot more work and isn't that useful in the short term. So if you try and do that refactoring upfront, you get pushback for wasting time on a small change. And then again on the next relevant change.
Developers often want to schedule these kinds of refactorings, because otherwise they get pushback on every chance they might otherwise have to fix these problems.
In your example, you don't refactor for fun, you refactor to integrate new things. Integrate the refactoring as part of the new work.
Guess what. The mechanic too has to deal with the old wheels, and don't get him started on the old screws that broke and the assembly that had to be cleaned just to work in decent conditions.
I always tell people to constantly refactor on a smaller scale and not ask management. It's just part of the job. A lot of the "big" refactors end up with just another set of problems but don't really make things better.
But I could be wrong too. I’m starting to think “devops” just means tech support with continuous rework.
Well, I think it's nothing more than giving the ops access and responsibilities to the same devs that created the mess to start with.