People who don't agree with you about it being a worse place to live than Earth in almost every regard might want to live there. I don't think Mars is a "worse" place to live than Earth in general, though it's certainly more challenging to live there. On Earth, humans have lived everywhere it's possible to live, not just where it was easy.
Antarctica was uninhabited until comparatively recently not because it isn't a nice place to live, but because without modern technology it's basically impossible to live there. The Sahara has been inhabited for thousands of years.
The Andes and Tibet are places which are so hard for humans to live in that the populations there have genetic adaptations to help them survive the low-oxygen environment. I mention that not to suggest that Martians might evolve the ability to breathe Martian are, but that humans haven't allowed mere inhospitability to stop them living in a place: history tells us that if it's possible to live somewhere, people will live there.