In addition, it's good manners. A person who is just contradictory to everything you say, and responds to all of your statements with a "no, you are wrong" over and over again with no further elaboration is usually considered an asshole and doesn't get invited to parties. People want to get invited to parties, so they try not to be rude in this way in real life except towards people they have utter contempt for and whose parties they don't want to go to anyway.
The anonymity of a virtual space however, opens up the doors to people to be out and out rude to one another. A downvote sans comment is even more anonymous because the recipient can't even see who their detractor was (and in addition, by the site guidelines can't even ask for clarification of the downvotes, which is silly).
However, HN guidelines also proscribe:
> Be civil. Don't say things you wouldn't say in a face to face conversation.
I think this is reasonable, my contention is that a downvote sans comment is the same as the hypothetical person who just responds "no, you are wrong" over and over again with no further commentary.