Very few people can afford to pay for homes in all cash up front. Plus, financing a home is usually a great idea, since they tend to be appreciating assets (as long as the interest rate is lower than the rate of appreciation).
If you earned $100,000 in a year but paid $5,000 in mortgage interest, government will not collect taxes on that $5,000, because you gave it to the bank. The bank gets money, but government doesn't: makes little difference if you gave it to the government first (who then handed it on to a bank), or you gave it directly to the bank.