There's a minimum karma requirement to be able to downvote. I don't remember exactly what the threshold is: it's either 200 or 500, I believe. See this thread:
I'm not interested in down voting anyone.
Edit: I've gone down 16 pts in the last hour. I know part of this is me being an ass initially, but clearly people w/ downvote ability are piling on...
Edit2: Yes, Me being an ass and idiot. Here'e the post that ended at 400
There is no other criteria for downvoting that I know of. When I achieved the threshold, the downvote button appeared. I didn't have to apply for anything, no one emailed me and said "We think you're swell! You get to downvote!"
I get the impression that you think there's some sort of secret down voting group. I haven't seen any evidence of that, either in behavior, or mentioned in comments.
It's fine if you don't want to downvote. There's no requirement to do so, and many people don't.
I've also noticed that you have been getting additional downvotes, and that bothers me because I want people to be able to have a positive, constructive experience on HN. I think what you're perceiving as piling on is a reflection of how strongly the community values good comments, not organized behavior. If you find that a comment of yours is being downvoted, within the first two hours of posting you can edit the comment (or delete it entirely if it has no responses). This may prevent further downvoting.
A lot of your comments have been on political or contentious topics. Those are particularly difficult considering how passionate and strongly people feel about them. I'm sure that's one of the reasons you want to participate in them, because you likewise feel strongly. And that in and of itself is good. That said, while there are some political discussion on HN (and indeed more recently reflecting current events), HN isn't primarily for political debate. I suggest taking a break from those discussions and spend some time in other threads to get used to the HN community and build up karma, taking extra care in crafting what you write. I believe you can lose enough karma (once your karma is significantly negative) to the point where your comments are hidden from most members.
I hope this is constructive. If you've got further questions, feel free to ask me, or contact the mods at hn@ycombinator.com. In my experience they respond pretty quickly.
I didn't delete it because if I make a mistake I own it. When I realized calling out another user was probably not right it was too late.
If you read through my comments you'll see that although I sometimes post on controversial topics, I'm not a troll.
Again, thanks for jumping in here.
Edit: but know, every time I reply to you I get down voted.
Edit: But huge props to gzpm Edit: No, I'm happy to talk to the mods. But really this has been not serious. It's about -20 now. Edit3: I will contact them tomorrow. But again I don't have much to stand on. I called out a user etc etc etc.
If you're not prepared to work within the community standards and framework, I can't imagine you're going to have a pleasant Or productive experience. I can only recommend taking extra care in commenting.