We're just not used to seeing the fronts of humans who have not rotated relative to us, so the mirror appears to have flipped the image horizontally.
If we customarily rotated around a horizontal axis, i.e. every time you look "behind" you're upside down (note this won't actually work very well given that you'd normally want to rotate around the axis perpendicular to the surface of the Earth, but just for sake of argument...), then we might believe that the mirror has flipped things vertically. It still hasn't actually flipped anything, it's just that we are viewing the front of a person who has not rotated.