To understand the significance, lets start by considering that the top 1% earn about 20% of the total income, and 60% of the lowest earners only have about 4.2% of the total income.
If you remove both those extremes, then average and median becomes somewhat closer to each other.