This is why one of my principles is to be skeptical of outliers. Often they are not real and therefore misrepresent the true data.
It's one reason median is preferred over mean, at the outset, as well as throwing out outliers just to see what things look like.