Downvoting here on HN does seem pretty arbitrary. (Either that, or I just don't post quality comments/posts :D)

I do know sites like HN attempt to identify patterns of voting like you described in order to remove abusive accounts, but I don't know how successful they are.

I could see downvoting requiring a minimal amount of feedback being a possibility, like you can only downvote if you are posting a public reply as to why, being a system that might have some viability.

As toxic as the internet can be, I am very happy the kinds of sites I work on don't allow anonymous users at all.