I agree with everything you're saying, but disagree with the reason.
The US federal government has to be too loose about keeping track of citizens specifically to avoid looking too fascist. One of your many American rights is to have no ID at all. Protecting that right for two dozen people makes everything extremely complicated for the rest of us.