Everything else should get ignored or warrants further discussion. In some cases it is even important to upvote "wrong" comments.
Same sort of deal when people are factually wrong, but aren't being a jerk about it: I don't think they should rate that highly, but I wouldn't likely vote them into negative territory either. And yeah, I might leave a comment too.
I always thought that the saving grace of /. was the metamoderation. People here downvote based on personal bias here frequently enough, and it would be nice to have the bad actors dinged for it. I'm not saying you do this, but it is done. Far better to have a highly upvoted correction.
Downvoting obviously wrong comments is very simple too. And useful. Like I said though, in the past I've tended to do so because it's incorrect, not to "punish" as I might with a mean-spirited comment, so I would never go past -1.
Naturally, the other end of that is a problem too, wherein people upvote for all the wrong reasons. I all too often see an comment that is wrong, but perhaps clever, and is upvoted highly. The most interesting aspect, I think, is that if you put a well-reasoned explanation of why the comment is wrong, you'll often see the trend reverse -- the wrong comment will start getting downvotes, and the correction will start getting upvotes. This might SEEM right, on first blush, but it also isn't.
The thing that irks me most though, is perhaps the hero-worship. I think if we were able to hide usernames until after we'd voted, then we'd almost certainly have better results. I cringe every time pg posts a one-liner that adds nothing to the conversation; Not because it isn't perfectly within his right to do so, but because I know it's going to be upvoted to the sky.