The second part of the system, is that, normally, no one customer will use more than one of the three load lines. That is why most houses only have one transformer on the pole out side their house. The power company makes sure they balance connections evenly within a given area. If they do this properly, the common 4th wire has no load on it.
Some buildings do use all three phases, but they would be running large industrial machines that benefit from having that type of power.
I tried to find the Wikipedia article that covered all this, but can't seem to locate it specifically.