I think you can downvote at some point, once you have enough karma