You need to have a certain amount of karma points to see the downvote arrow. I think 500 so you're almost there.