Not necessarily. If said refactoring requires very little work, then sure. But, as it often is the case, refactoring often takes a fair bit of work and probably more than expected. Thus, it is out of scope of the original PR/branch.
If the PR is for adding a feature, then focus on that. Refactoring is better served by having its own dedicated focus and branches, respectively.