We're not naming the Dyson Sphere after him, then. ;)
But I don't think his objection is using solar power for space-based infrastructure. Quite the contrary! (Dragon is solar-powered, but could fairly easily do everything except maybe long-duration Dragonlab missions on batteries if Elon really didn't like solar). I think the objection is to the complexities and transmission losses of beaming orbital solar power back to Earth.
Of course, if you start moving your energy-intensive industries out of the gravity well...