If you opt to have "showdead" enabled in your profile you'll still see flagged comments but marked as [dead]. In this case you can also vouch for comments in the same fashion (or add another flag to make it harder to be vouched for).

I think it's something like >30 karma to flag/vouch posts (so you should well qualify). This means users actually gain that well before the ability to downvote... but the site sure does a good job of hiding that from them lol.