Making code changes means one takes on the risk of breaking things. I think the people who don't break stuff are the ones who continue to refactor. The ones who don't either are scared of breaking stuff or had a bad experience with breaking stuff.
I suspect the people who refactor have what can be describe as perfectionism or obsessive compulsive.
I'm in that bucket. I can see certain code which is too complex or didn't handle a requirement well and I want to get it to be stated more simply.
You either didn't fully read my post, or didn't understand it. It's not that you can't, it's that it's a bad idea. I would instantly reject any pull request that wasn't compartmentalized to the issue at hand. It's an organizational nightmare to handle both features as well as refactoring in the same PR.
But obviously to each their own.