I know that. I've talked with people who work on banking systems who have confirmed it.
It's true there are other reasons for the limits, such as the ones you listed, but if you are with say Bank of America, your limit and my limit can be different. The more credit history you've built up with the bank, the higher the limit will be. This is mostly because they trust you more and therefore consider the risk for defrauding smaller.