Walking with groceries in the rain won't kill you, I promise. It's a small inconvenience but in my opinion you get a lot back for it, everything just becomes more village like and friendly. Being in a car does not tend to make people more friendly.
And just because there are nice places to hang out does not automatically mean there will be drugs and crime. That is not my experience at least.