There are many levels of analysis to a paper. Yours is part of it but not the full picture. One other thing is the novelty aspect. To assess that, you need to know the current state of the field. Also the best practices, known methodological pitfalls, judge what is a valid chain of reasoning and what is too speculative etc. It's far from simple and I often see outsiders online criticizing largely irrelevant things while missing core issues. There's no shortcut to expertise in a field. You need first hand experience and knowledge of the state of the art. Otherwise, if papers have to talk to laypeople or generic scientists, we'll see even more smokes and mirrors. The idea of peer review is that a work should be challenged by those best equipped to challenge it.