SF is a very densely populated place, so that shouldn't be a surprise (if you live in NYC, I don't believe you).
So have I - and downtown SF is worse (in terms of littering, human excrement, aggressive panhandling, street crime) than the problematic parts of NYC, DC, Philly and Baltimore, worse than densely populated capitals in Latin America or East Asia or Eastern Europe. It's better than Indian cities but that is a very low bar for one of the world's richest cities.