My bet is that people will start posting a comment saying "upvote this comment if you want to downvote", the comment will rank first and it will be as if you have a public dislike count
* "the information in this video is wrong"
* "i don't like the person who made this video"
* "this isn't the content i expect from this channel"
all of which is vastly more useful than just a downvote.
Unfortunately comments are less indelible (more delible?) than downvotes. As evidenced by the shadowban in OP.