As long as fact-checkers can be sued for incorrect and malicious fact checking and fact-checking isn't grounds for censorship and information suppression, I would be on-board with you.

The most that fact-checkers do when they are egregiously and utterly incorrect is a correction to an eye-rolling "needs context" label.

The standards of objectivity of fact-checkers (who are all left-leaning partisans) are ridiculously slanted. One set of strict standards for my opposition, another set of lax, wink-wink standards for myself - modern day "fact-checking". Any criticism of the same that becomes popular results in being banned.