It's not just the downvotes. They don't help, but it's also the responses.
There is a major psychological effect from getting attacked for stating an opinion you might hold to heart. As an avid reddit/hn commenter, I feel it often.
I don't really care if someone disagrees with me on something minor, but if it's something I truly believe is important and I see people disagreeing with it left and right and downvoting anything in line with that opinion, it makes me feel weakness and despair.
Weakness: I am overwhelmed by the people disagreeing with me. I can't answer everybody. Not because I don't have the arguments but because it's pointless, won't lead anywhere and will achieve nothing but make me look insane. The "hivemind" effects makes widely-held opinions even stronger and minority opinions even weaker. Alone, I am powerless to counter that.
Despair: Let's say someone thinks gays/black/women/whatever shouldn't [have some human right]. You truly believe that's wrong. If someone says to me "I think women shouldn't have the right to vote" and is impossible to convince, I feel pretty awful about it. I feel like that person is contributing in making the world worse, and I live in that world.
Now what if that's not about some minority-held opinion, but about something far more widespear? What if it's, for example, similarly insensitive and disgusting comments about islam/muslims? I don't just feel awful being around that person, I feel crushed by the amount of people who would agree with it. I'm quite afraid of what happens when such opinion is widely held. I feel like I'm looking at a lifetime of awful and I feel crushed by it.
My fiancée is a muslim. This is empathy kicking in. Not everything affects me that way, but I'm far less likely to comment on something I don't hold to heart.
But yes, I do enjoy HN because the hivemind effect is far more limited. The hidden downvotes and sorting algorithms are a million times better than Reddit's. It's not ideal, but it's still an excellent place to have discussion and a "good enough" place to have debates.