But your property rights are not absolute. We don't allow people to dump toxic waste or burn tyres on their land, even if the land, the tyres and the toxic waste is entirely theirs. YOUR prerogative ends somewhere. I know indidualism is a tantalising idea but we live in a society with rules.

People collectively can decide(and frequently do decide) that the values important to a town, a city or an entire state are more important than your right to buy and leave property empty. There are many places which either forbid owning property entirely if you are a foreign national and don't actually live there, or impose a heavy tax on such purchases. Vacancy tax is a thing too.

Basically if a city(and by that I mean people of the city and its representatives) don't want houses to be sitting around empty, there are laws that can be introduced to force people to either move in, rent or sell the place, or they can keep the place standing empty at a heavy financial penalty if they wish. Like I said earlier - your prerogative ends somewhere.