Yep, but there you are refactoring as a necessary prerequisite to achieving a clearly-defined end goal. That’s part of the development roadmap and should be planned and budgeted accordingly. When building a house, a professional builder knows to dig out and pour a solid foundation, so that by the time they get to tiling the roof the walls beneath it aren’t already sinking and tearing apart.
That’s very different to just dicking with the company codebase for one’s personal amusement. That’s the bloke with the dozen rusted automobile shells sitting on bricks in his front yard, while he’s in his shed “busy working” on number thirteen. He’s not productive, he’s just playing with himself. And making the whole place look like trash while he’s at it.