Depends on what you call a difference. The nice lady at the DMV, or the IRS or the county assessor's office will likely know far more about you than you ever will about her because of access to data. A lot of that data is not publicly available.
I have no reason to trust them any more than I trust anybody else. Saying, "They won't misuse the data because it's their job not to," is as comforting as "Nobody will misuse the data because it would be illegal."