I'm not even sure if it's possible to pay most bills in cash in my country. I have had a bank account frozen due to identity theft; it took me less than 10 minutes to open a new account and get a new debit card in my Google Wallet. The bank transferred my funds within a couple of hours and all of my recurring payments were automatically switched within a couple of days.
Maybe my experiences are atypical, but my impression is that consumer banking in the US is quite dysfunctional.