I still don't get why folks don't consider living in places just outside of SF that are much more affordable. Places such as South San Francisco, Daly City, Brisbane, Oakland, etc. The proximity to the city is very close (5-10 mins).
And that doesn't include walking or waiting time.
Driving might seem like a plausible option, but that's generally worse. Oakland City Center to Embarcadero is 25 minutes by car, not counting parking. Daly City to the Mission is 20 minutes. And that's not counting parking, which is never easy in SF.
People do live in those places. But it's definitely living in another city.
I live/work in soma and as long as I avoid commute traffic, Daly City really is only ~10 minutes away. A coworker lives in Brisbane and his commute averages 11 minutes. I frequently visit parts of Oakland far further from City Center (say Mills College area), and it is just under 20 minutes.
If you lack a car, well, the numbers look terrible as well in San Francisco. This is most obvious when you note that you can bike from most A to most B faster than Muni can take you. As another example, downtown Oakland is closer to Embarcadero or even Union Square by BART than Inner Sunset is by Muni Metro.
The "as long as I don't drive when most people want to drive" thing is nice if that works for you, but it's definitely not the common case. Most people have to commute when most people have to commute.
I agree driving to the Moscone sucks (driving on 3rd past Harrison is a nightmare).
Things are much better if you are talking about the Mission District: https://maps.google.com/maps?saddr=Daly+City,+CA&daddr=M...
(I mentioned "avoiding commute traffic" as I figured "place you want to be" meant for fun, not work.)
Or on a week day, when the parking lots are beyond full?
If you're into drinking, even socially, it means no designated driver. No hunting for parking (can take > 10 mins in SF), especially in the areas where all the food and bars are.
Most neighborhoods in SF (and I'm guessing NYC) there's usually some shopping district with a street full of stores and restaurants, non-national chain, within walking distance.
Growing up in Orange County that is something I had never experienced and at least until I have a spouse and kids I don't really want to go back to.
Where I live I'm within a 3 minute walk to a Whole Foods, 15 or so restaurants. A convenience store. 3 cleaners. A twice a week farmers market. Several banks. A drug store. 6 bars. 5 coffeeshops. 2 bookstores. 2 bike stores. And I'm only a 10 minute walk from the mission or a 5 minute walk to the muni (to go downtown).
Compare to the O.C. where pretty much had to drive everywhere. Daly City is a similar place.
That said, SF is more of a place to live if you don't have children (and if you have children, you better be rich). Those with kids tend to live in the suburbs.
The ferry is basically worthless if you don't actually live in San Rafael. It's a nice ride, but very slow, and not cheap.
Personally, I quite like the bus system: if you don't mind taking the bus, Golden Gate Transit is great. The commute from somewhere like Novato is still long (1hr) but you can use the time to sleep or listen to audio books. Be sure to check the schedules of the express routes you care about: they end kind of early (~6, 7pm) and the non-express routes that run longer take easily twice as long.
Driving is a different story. There are three things to consider: 1) leave early, like 5ish, especially if you're north of marin 2) if you work in the financial district, the time to drive from the edge of SF to parking is substantial and 3) you can park in the financial district itself, but this will cost you around $25/day.
I suppose there's a fourth thing to consider re: driving which is that your drive home is almost guaranteed to be painful if you have to go beyond San Rafael and are not car pooling.