It's happened to me though; one scenario is that you created a couple of merge requests for the same repo over the course of a day, and the next day feedback from code review or testing comes in. You end up having to switch to various branches to fix the reviews.
But that's really not that big of a deal. The context switch is worse than the git checkout call.