Comment by jsprogrammer | Hacker News Reader

Comment by jsprogrammer | Hacker News Reader

Parent Full thread

A scientist should be able to describe all assumptions and manipulable variables that go into an experiment.

>It's reasonable for a scientist to ask for a certain level

Maybe, but a scientist should be able to specifically describe what that 'level' is.

Instead, this article seems to present many individuals who think appealing to an abstract 'level' of their own choosing and to which they seem to not be able to describe sufficiently to anyone else (which is fishy, because all of this serves, intentionally or not, to prevent anyone from attempting to replicate their experiment).

In my view of science, there should never be a case where 'misgivings' about a replication attempt should ever be expressed. Either the experiment will be accurately replicated or it won't, in which case we can probably determine if the replication was not accurate or if the original experiment was flawed (either in design, execution, and/or analysis).

Science gives us the tools to remove things from the influence and design of opinion and to examine them from viewpoints that are as free from subjectivity as possible. Once you start moving into the realm of opinion (expressed misgivings) you are leaving the realm of science and moving into philosophy, religion, or worse.

I disagree here, because the inevitable result of a highly visible series of failed reproductions is a big media hit. The scientific community may be able to poke holes in the reproduction attempt and sort through the damage, but the media and the court of public opinion certainly can't. Not until long after the reputations of possibly faultless scientists have been ruined irreparably.

So it's important for the reproduction attempts to be as high quality and rigorous as we would hope the original studies were. And it behooves scientists to make sure that these attempts are legitimate, unbiased and equitable, and to investigate any experimental flaws and biases of the experimenters before the results.

Misgivings about a reproduction attempt don't indicate denial of the validity of the scientific process, but recognition of the volatility of scientific news media. The unfortunate reality is that both sides of this effort, both original researchers and reproduction attempts, are subject to a great number of biases and restrictions. Subjective opinion does deeply affect the lives of scientists, and it's not possible for even the best scientists to live in a bubble of scientific purity and assume things will work out.

>I disagree here, because the inevitable result of a highly visible series of failed reproductions is a big media hit.

Interesting proclamation.

If a result can't be replicated after many 'highly visible' attempts then the result should be called into serious question.

>The scientific community may be able to poke holes in the reproduction attempt and sort through the damage, but the media and the court of public opinion certainly can't. Not until long after the reputations of possibly faultless scientists have been ruined irreparably.

This sounds like unwarranted fatalism to me. If the result was not reproduced because the experiment was not actually reproduced...I don't see what the issue is here.

>So it's important for the reproduction attempts to be as high quality and rigorous as we would hope the original studies were. And it behooves scientists to make sure that these attempts are legitimate, unbiased and equitable, and to investigate any experimental flaws and biases of the experimenters before the results.

This is why it is critical for any researcher that desires credibility (and more importantly: explanatory power) to detail their work accurately enough that someone else can exactly replicate their experiments in order to provide independent verification of their claims.

The best way to ensure that replication attempts are 'legitimate, unbiased and equitable' is to ensure your work is good enough that someone can actually (as opposed to merely attempting to) reproduce it.

>Misgivings about a reproduction attempt don't indicate denial of the validity of the scientific process, but recognition of the volatility of scientific news media.

Maybe.

>The unfortunate reality is that both sides of this effort, both original researchers and reproduction attempts, are subject to a great number of biases and restrictions. Subjective opinion does deeply affect the lives of scientists, and it's not possible for even the best scientists to live in a bubble of scientific purity and assume things will work out.

This sounds like something scientists need to work towards resolving.

Reply on news.ycombinator.com