Based on your comment I think you may be missing one aspect:
The money you get lent at the bank is the money people are investing.
Both of the things you describe in this comment are two halves of a market. There's someone borrowing money and someone lending money. The borrowing becomes more expensive because the lender has better alternatives.