I think you're thinking of somewhere else. The up/down votes are a way of agreeing or disagreeing without cluttering up the comments with a bunch of "me toos" or "nuhuhhs"
Probably the most that I've written about downvotes on HN is at https://news.ycombinator.com/item?id=9440694.