Are you sure about that? I’m not sure you are hearing and understanding my suggestion. A review would always consist of one paper being ranked above another one. The tone of the review comments would have no bearing on the usefulness of the review. The reason we even have critiques in reviews is to justify the score that the reviewer assigns. Reviewers are specifically asked to produce reasons. What I’m suggesting is different because reviewers would rank two papers against each other and not be expected to justify their decision, nor would they be asked to provide a magnitude for their opinion. Comments and critiques would be allowed, but there’s no need for harsh reviews.

I’ve seen a lot of overly and unnecessarily harsh reviews. Anonymizing enables over-stating criticism, it happens routinely. I don’t think I agree that harsh critique is necessary for a healthy review system. It is already the case that good reviews are not extremely harsh, they focus on the facts and are willing to stand by their statements. I don’t personally know that many researchers who have trouble being direct in person and face to face, or of offering constructive criticism.

> As a reviewer you are only advising the editor

This completely depends on the journal or conference. Quite a few of them, especially the larger ones, do not override reviews casually nor often. And what I’m suggesting is a system where this idea can change, where editors can more easily trust the review results, and won’t need to override the decision.

> People are petty.

This might well be true. And so I’m not entirely understanding your argument. It seems to be simultaneously suggesting we have a problem, and defending the status quo as the way it needs to be. What would you suggest as a way to improve the review system so that pettiness has less influence than it does today?