Are you saying you'll be more reluctant to take away flagging rights than to take away commenting rights? Or do you feel that everyone who's been shadowbanned has in fact repeatedly misused their commenting rights?
That's a little uncharitable, considering this feature is acknowledging that the moderators can make mistakes.
Best wishes.
It seems to me the whole point of the vouch features is to have an easier recourse for other commenters to respond to people being shadowbanned (or otherwise dead for mistaken reasons, like tripping a filter or unfairly flagged by other users), rather than actually having to find the mod email and send an out of band message.