I'm not a libertarian. I believe that I have a duty to pay taxes, and I even think tax rates (for me at least) are fair, and I have no problem with redistribution. But I also have rights, for example to live where I like so long as I can pay the rent and can find someone to rent to me. My impact on other people through rent prices is not an externality and does not need to be controlled by the government. A free country is a country where you can engage in private transactions as you please, not a country where everything is done with permission of the community.
Also, a lot of the animosity is fueled by something other than pure economics. If you read these kinds of articles, the complaints tend to morph from "rents are rising" to "they aren't part of the community" to "they are socially awkward white kids". The idea that political action should be taken against a group because other people find them unattractive or dislikable is sickening to me.