If enough people upvote purely for agreeing, it becomes possible to get karma for vapid posts that pander to the community.

I think it would be better to just factor in agreement as part of whether you'd like to see more. Maybe you agree with something, but the sentiment has been expressed so many times that it's getting tiresome.

Actually:

Upvote = "I'd like to see more comments like this" || "I agree with this".

Downvote = "I'd like to see fewer comments like this" || "I disagree with this".

the problem with downvotes is that the parent of this comment being downvoted is more likely to be because someone didn't know what || means rather than because someone wanted fewer comments suggesting minor, helpful improvements on the parent idea.