Whether it's only a minority that takes a certain view is not just important: it's fundamental to a society's set of morals, which is what the question really is about.

What is a culture if not the rules and customs a large majority of a certain population agrees on?

Some cultures try to accomodate minorities, but there's always a threshold to what's tolerated even in the most open societies. A certain minority may not like something the majority does, but unless there's very little cost for the majority to stop doing that, they won't, even if that offends some groups.

There's no way around this because there will always be groups who take offense in things you may consider completely and utterly harmless (I gave the example of Australia aborigines not being comfortable with talking about or depicting deceased people - obviously a problem in the age of movies and books). A society that tried to acommodate every group's sensibilities would be completely unable to function.

The more distant the groups, the more patently obvious this becomes. If you are not sure what I am talking about you probably should try to learn more about other cultures.