"The City" refers to the core of whatever urban area you're in. It's meaningless outside of your local frame of reference. The only place in the US that has a halfway legitimate claim universal ownership of that name is NYC and that's tenuous at best.

The only people who seem to get annoyed with the term "San Fran" and Frisco are either transplants or hipsters(which are usually transplants to begin with). Some of the old schoolers and hip hop community still use the term Frisco.

Only _some_ people from the city proper frown upon "San Fran", frisco, etc. Most people don't care and more over people that I know from elsewhere in the Bay area do some times call it San Fran or Frisco, if only to annoy people who get annoyed by the monikers.

I couldn't care less what people call it. When I lived in towns closer to San Jose, 'San Jose' was the reference 'city'.