But there have been many successful start ups through out Atlanta's history, such as Coke, Turner Broadcasting System(CNN/Turner/etc), UPS, and more. All of these were revolutionary start ups at the time, and now are massive corporations. I expect future companies like this to be founded here in Atlanta, because it is essentially the capital of the South East, geographically about a quarter of the country.
What makes me want to move away from Atlanta is the poor city infrastructure, the dependence on suburbs, and the lack of culture. In Atlanta there is possibly less muse than in San Fran or NYC, and the city as a whole can sometimes come across as anti intellectual, which is quite the opposite of most of San Fran and NYC. Beside GA Tech and Emory college students, networking would be relatively small when compared to NYC or San Fran. But for someone who already has their network built out, such as a start up from SF/NYC trying to cut costs, Atlanta could be the place. And it's also ideal for people moving from smaller cities who don't want the large city feel.
If Atlanta had a more walkable and livable environment, it's be quite amazing.