Lol you're not even being a jerk about it and this still gets downvoted.
It’s not necessarily being used as a disagree button in such a case. It’s similar to, “This might be unpopular but ...”; some people will principle-downvote (right or wrong) for what they see as conducive to a bad discussion.
This place has an in group, I see… serves me right for typing comments on a VC blog
Stay a while longer, get a few more upvotes on your comments (the general key is to add substance), and one day you'll get the option to downvote comments | flag topics / comments, etc.
Downvotes and more are unofficially documented here.
Mostly I agree with your take, my assumption is that the karma requirement for downvoting is to help prevent downvote spam, but that's just me making crap up.