I am not a philosopher and I have no idea how to reason about situations where individual rights for individuals and groups of individuals clash.
That's a great question. In short, the 99% of people should tolerate the 1%. They don't need to choose to associate with them, but don't take action against them to silence them.
> I am not a philosopher and I have no idea how to reason about situations where individual rights for individuals and groups of individuals clash.
I think this quote is probably appropriate:
"If all mankind minus one, were of one opinion, and only one person were of the contrary opinion, mankind would be no more justified in silencing that one person, than he, if he had the power, would be justified in silencing mankind." - John Stuart Mill
In that case you should understand that no one is obligated to hear you whining about topics, specially those you're not even willing to discuss.