During regular Monday - Friday business hours, the city's population surges by at least 20 percent -- or more than 160,000 people. https://www.kqed.org/lowdown/11327/interactive-how-san-franc...
that number could be wrong. but that 20% beats your 4% by a comfy margin. and that 20% has special impact on the city center that employs many service workers. those people are no longer commuting. if they're no longer commuting, then prices of apartments close to jobs will lose value. then rents will fall. rents are hard data that you apparently refuse to google.