From my unresearched perspective, the difference is that Musk (admittedly) is pursuing crazy ventures because he's passionate about them -- and oftentimes for no other reason. He apparently originally expected that SpaceX would be broke by the time he had managed to get to Mars.
Branson seems to be deeply rooted in a business-first approach to creative innovation. "Yeah, we'll do it, but if the market accepts it."