You actually can't downvote submissions at all. Your only option is to flag it.