Another common occurrence that makes me mad is people who want to go UP entering an elevator that's going DOWN, when the downward traveling elevator stops due to some people (coming from the upper floors) exiting it.
Of course, I think everyone's probably done it at least once simply due to inattentiveness.
What's the problem?
Overall it's the same amount of stops regardless whether you take the right or the wrong one. But for some reason being in an elevator that stops without anyone getting in or out irks me.