It makes me want to stop and do something else, something more pleasant. It evokes a nasty image of shovelling shit, where I want to think of what I am doing as polishing a raw gem.
Furthermore, to me it seems kind of disrectful to numerous code authors of such 'smells'. Maybe they didn't know better, maybe it used to be 'clean' but they were in a hurry to fix something. Maybe they made a different trade off. Labeling what they did as 'smelly' seems impolite and unnecessarily judgemental.
Objectively, it also narrows the perspective: something that smells is harmful, stale or poisonous. It affects disgust and moves us away from it. Food that smells has gone 'bad', which implies it is not good (anymore). So to fix a smell, is to move from bad to good, which is a very binary change. It's either good or bad.
But I think with code this not always the case, and even when some code is clearly better, seeing improvements as more appropriate or efficient tradeoff is almost always much more helpful.