Elon has used those loans to finance a rather extravagant lifestyle. He own FIVE mansions in Bel Air, all right next to or near each other. He owns a $70,000,000 private jet, a top-of-the-line Gulstream G650 ER, and flies everywhere with it, burning immense amounts of fuel sometimes just commuting within Los Angeles (between Hawthorne and Van Nuys airports, for example). Because his unusual compensation arrangement at Tesla is contingent on the company's increasing market value or maybe someday turning a profit (which has never ever happened), these loans against his Tesla stock are his main bread-and-butter.
In other words, Tesla's fate as a company and its stock price are intimately tied to Elon's personal financial solvency. That's gotta create some conflicts of interest between his various jobs as CEO, head of the Board of Directors, and a person who doesn't want to be personally financially wiped out.