I loathe telling people I'm a vegetarian for this reason. People think that my decision to not eat meat is an attack on _them_ and not a choice I've made for myself.
Edit: a word
Edit: a word
When someone says "I don't drink"—an entirely valid and acceptable choice!—at a point in time where we've already established that someone else in the group does, it draws attention to that disharmony. It's not that not drinking is the bad thing, it's just the delta between that and the previously accepted reality that another in the group does. You've subtracted some territory from the common ground.
To get a sense of this, consider an imaginary scene:
A group of people are standing around trying to decide where to go out. The first few people suggest a place that is does not serve alcohol and that they are OK with that since they don't drink anyway. The next person mentions that they do drink.
In this case, it's now the drinker who looks flatfooted, at least as I imagine the scene. To me, then, this indicates that it's not being a non-drinker that's socially awkward, just that expressing a divergent preference that is.
Fortunately, most people have tact and can gracefully handle a little dissonance here regardless of which preference comes up first. Certainly, in the groups I frequent, we try very hard to make non-drinkers and drinkers feel comfortable and included regardless of who makes the first move.
The way this tact gets expressed is through plausible deniability. We express our preferences as vague or potential so that if a clashing preference comes up, it's easy to retreat from ours and get back to common ground.
That's why, at least in American speech, people say things like, "I could go for __." "I would be up for __." "__ sounds good." All of those are softly worded enough that they can be amended later to reach consensus without losing face.
Person: Whatcha' getting?
Me: I'm not totally sure.
Person: How about the steak?
Me: No, I don't really want that. (oh no.)
Person: Really? It's supposed to be really good.
Me: I hear that, but I don't think so, not this time.
Person: Why not?
Me: Well, I don't eat meat at all, actually.
Person: Oh yeah? Are you vegetarian or vegan?
Me: Vegan.
Person: Oh, I'm so sorry to hear that. The steaks is gonna be so great. Don't you just hate how pushy vegans are? They're always going on and on and on about...
This story happened over and over and over and over and over and over and over to me.
It was the same with not drinking, too. "Why did you only get a Pepsi?"
There was a girl in my dorm during freshman year of college who would literally start sobbing if she saw anyone eat meat.
There are always going to be outliers like that girl, but compare the relative frequency of seeing people like that vs the absurd number of bacon products (jelly, soap, toothpaste).
People like that might make up a small portion of the vegetarian/vegan population, but they are a very vocal minority, which skews perspectives.