1) No check digits--every number is valid. Adding a couple of extra digits would help tremendously
2) No ability to retire/change the number--this is the big one. Once the criminals get your SSN--game over. You can't change it and shut the old one down. If you simply changed this, a lot of the issues with SSNs would go away.
Then there are the usage issues:
SSNs have two tasks which need to be separated:
1) Identification for "blocking" or "association" tasks. SSNs are fine for identity on a blocking task--ie. trying to shut something down, taxation identification, etc.
2) Identification for "allowing" tasks. SSNs are BAD for allowing tasks--new credit card, new bank account, etc. These kinds of tasks should always require SSN plus something else. This is where the primary failure modes are.