I used to feel the same. But in reality, we prove our identity daily - every credit card transaction, banking, electronic tolling, bill payment, health care visit, tax payment, legal proceeding, employment opportunity, voter registration …
What protects us from invasive search is not lack of a uniformly accessible system for identification - its due process. And if the government chooses to compel you against your will and without due process - whether or not you have a laminated ID in your pocket will be irrelevant.