Oddly enough, here's a situation where headline's idea sounds better than what's discussed in the article.

The idea of a "confidence detector" seems both Orwellian and AI-complete.

But a "bullshit recognizer" seems simpler and practical. A device the recognizes contradictions in text, conclusions unrelated to evidence or similar "egregious" things could be useful.

It might even be a trainable thing. You have N papers with M passage-combinations that show the claims of said paper are bullshit and develop a finder/recognizer. The problem could wind-up not that different than sending Watson at Jeopardy.

The drawback, of course, is the thing would have to be taken no more seriously than a grammar checker - it just serves to point reviewers at potential problems rather than serving as proof of a problem.