The State of New Jersey actually sued a Florida pizza place for trademark violations: https://www.nbcnewyork.com/news/local/New-Jersey-Loses-Trade... they lost, however, because nobody ever confused a Florida pizza place with a New Jersey highway.
Under the name "The Postal Service" and within US borders? Yes, they do think that, and their trademark would agree with them.
The bizarre part is trying to apply their trademark to an area of business that is not related to what USPS does. USPS does not play music gigs. There would be no danger of someone confusing USPS with a band. Trademarks are also limited to the line of business that the trademark holder operates in. This is why Apple Computers and Apple Records (the record label for the Beatles) were able to operate without trademark conflicts for decades until Apple Computers started getting involved with music and then needed to come to an arrangement with Apple Records.