In reality, there’s a balance between moving fast and and moving slow. It’s difficult to communicate that balance because every type of product demands a different balance. I suppose that intuition comes from experience, which is a terrible answer for someone trying to learn.
I'm guessing this continuum also depends on the nature of the business one is in. In a fast-moving, consumer-oriented world like social networking, "move fast and break things" is probably very good advice.
In a slower-moving, enterprise-oriented world, "move slowly and steadily without breaking things" is probably the right orientation. Those kinds of companies probably die in fast-moving consumer land.
A while ago I read an article about the programming teams that work on the space station, on satellites, and that sort of thing. Alas, I cannot find the link right now, but the gist was that those teams move very slowly and work very hard towards correctness. Any bug found is generalized and their entire code base scoured for similar bugs. That's a different environment than social networking.