It's not just that you can't fly into busy airspace, it's that you can't fly into airspace where you need to talk to an air traffic controller for VFR flight (class B-D).

If you live in or near a big city chances are that the airports out of which you will want to fly will be close to airspace into which you cannot fly. It a hassle if you have to drive out to the boonies in order to start your flying trip or if you have to take the long way around a large block of busy airspace.