Don't most social media sites ask for permission to upload your contacts from elsewhere? The answer (as Nancy Reagan taught us) is to Just Say No.
Yes there is (a lot!) of job-hunting spam on LinkedIn, but when you need a job the spam can help. Even if you don't, it's useful to have a public place with an email address for professional contacts to find you, in case you switch firms.