It doesn't matter if one person owns 20 properties in a neighborhood to engage in rent seeking behavior or if 10 people own two properties. The result is the same. The problem is that the market is broken, there is a shortage, and the shortage is perpetuated by cartel-like behavior by the local homeowning electorate.

Given that we probably won't fix that, I think that if you want to operate a B&B, you should be allowed to operate a B&B, but if you are going to get out of the business of operating a B&B, you should have a cool down period of 3-5 years (enough for the operator to take on real market risk).

That means that the business must only be in the business of operating as a B&B and until that cool down period ends, and then can be converted back into a residence. This is a non-trivial amount of time, and it means that people in the business of short-term rentals can't necessarily just causally operate them, because if there is a housing slump, they won't be able to exit their position quickly.

This is a massive disincentive to folks trying to operate a B&B to facilitate asset appreciation, but is not a significant impediment for people who want to operate a B&B as a going concern.

This solution invites all kinds of problems, like black and grey markets for housing, but that's because this isn't a real solution. The real solution is to just legalize density and let people build as many residences as they want on property they own. It also helps to have a property tax that incentivizes maximal utility of real estate and disincentivizes hoarding.