The key thing you've said though is "comments that provide value", a metric by which both up and down votes are important.

In the perfect world, I'd come to HN and the top 2 voted comments would be the most succinct and well argued sides to the discussion (assuming there's a traditional for+against), then the top voted comments the tier down would be the best refutations of those points. That way I can read both and get to my own balanced view quicker. If all the voting system tells me is "this is what the average HN reader agrees with" that's far less valuable.