butter is one of the most easily substituted things around and most vegans i know don’t care about honey.
definitely more restrictive but i think the dairy/cheese thing is the big one
people also feel that vegans are judging them morally and people are very defensive about their eating habit, hence the dislike