"Owning a home in a crowded city, not living in it the vast majority of the time, and refusing to rent it out to people who desperately want to live there imposes a negative externality on the city"
It is proposed that doing so imposes a negative externality.
That proposal is not without counterpoints, however.
Specifically: that the owners of those homes not only pay their full property taxes already, but they impose fewer costs on the infrastructure of the city while not being present. They don't drive on the roads or walk on the trails or use the public restrooms, etc.
Look, I proposed a hypothetical situation[1] that has nothing at all to do with Chinese speculators buying up housing assets in a safe haven market. I proposed an unintended consequence on an otherwise innocent, well meaning actor. It's very interesting to note that the response is, basically, "too bad for them".
[1] I don't live in Canada.