I found out about this after having three different people in a weeks time knock on my door and ask me where the fuse boxes are. Not a huge deal if my neighbors do it, but these are total strangers. At least when someone moves in to my building they have multiple background checks. And since full time residents live here they have a reason to be polite as they see me daily. They have some connection to the building and its residents.
A few weeks after the start of this I was woken up at three A.M. by my dog barking at the fire escape. There was someone on the fire escape looking into my bedroom window! After calling the police I found out that it was a renter from the unit below who wanted a better view and decided to peek into my unit at the same time. This has happened half a dozen times in the last year.
I work typical business hours. People who stay in the unit as guests are generally on vacation and this can lead to loud parties that last all hours of the night. I have a few neighbors who work strange hours and we work out noise agreements, you can't do that with people who are staying one or two days.
There's also the moral issue. It's hard to find a place in San Francisco, what they are doing is illegal, against the terms of their lease (it's very hard to evict tenants in San Francisco, the owners are trying), and they are denying someone a full time residence so they can make extra cash.
I like the idea of Airbnb but I agree that the people who rent units need to be held to some standards. A permit that neighbors can contest seems reasonable if it's going to affect their standard of living.