You need to accumulate a certain number of points (by posting comments or stories that get upvoted). Then you get the privilege of downvoting.