> Of the ones who vote by agreement, some will agree and some will disagree, and they end up cancelling each other out.
It's possible for this to happen, but I don't think there are that many hot-button issues where the two sides are exactly balanced. More commonly there will be at least a slight skew in one direction or another, which gets magnified in absolute score the more people there are who vote based on agreement. For example, if there's a 60/40 opinion skew among people who vote based on agreement on a particular topic, and 20 people vote that way, you already have an 8-point rating gap between the popular and unpopular comments (+12/-8 versus +8/-12). On really high-traffic topics, where 100 people might up/downvote based on agreement, you only need a tiny skew from 50/50 for the votes to pile up in one direction or another.
I think in practice that often swamps the quality-based voting, especially on high-traffic articles: you sometimes see extremely short and not particularly insightful comments that express a popular sentiment up at +40 or more.