So, the "tyranny of the majority" is an illusion of those who are highly intolerant?
There are accepted norms which are accepted by the majority for historical reasons, but also because people believe the norm seems both honest and accepted by many of who they see as honest people who are similar to them. So the norm is not just something you agree with, it's something consider to be decent and can be consented to.
In society there is always one big dominant river which current is going in one direction. That's the social contract. If you don't have one rule for everyone, there is no society.