>A US passport card doesn't cost much

That's the problem. It doesn't cost much, but a Social Security number costs nothing and takes nothing to get. Something is infinitely more than nothing. Passports also require a lot of paperwork and identification to get that people may not have.