Hacker News Reader

Comment by analog31 | Hacker News Reader

The replication crisis is serious, but it's not the collapse of science. Maxwell's equations still stand, for example. You can still identify chemical elements from their spectra. And so forth.

That is not to diminish the problem. Over-reliance in purely statistical methods has gotten us in trouble -- in some fields more than others. Probably the main issue is that the fields affected are ones that happen to the biggest right now, with potential impact on quality of life and public policy.

Fields that are doing a better job of weathering the storm, such as physics, have tended to have a larger tool belt, and the use of statistics is kept in check by other methods such as the development of theory, and the ability to study problems from multiple vantage points. Also, physicists can choose our battles, and only look at problems where more robust statistics can be used.

View on HN

Even in the fields most affected, I'm not aware of any knowledge generation efforts outside of science that are going better.

It might just be that the social sciences and medicine are really hard, and our best efforts fail often.

Edit: I mean the best effort we could reasonably make as a society, naturally there have been individuals and groups who have been doing things they knew or should have known were wrong for their own reasons.

> It might just be that the social sciences and medicine are really hard

As one field often builds upon another, the so-called soft/social sciences could also be called Higher/high-order sciences; and the complexity progression might not be linear, since at the high-level these topics tend to merge such that they are no longer layers but rather aspects of the same societal mega-graph e.g. economics is the study societal economy, political science is the study societal politics etc, all of which are part of the same system, and hence inter-react, in the same way a study of volcanic activity, the tides, the magnetic field, the atmosphere, weather and the terrestrial water table/system are all inter-related topics of the same planetary model (I expect all these things to become distinct topics of study too, at a greater scale if they aren't already).

Another consideration; Natural versus "human" sciences - any system that involves humans (any study of an aspect of human society) differs from the study of natural processes.

1) the scientists, or scientific institution is a part of society,

2) humans can react to scientific investigation and inquiry - a natural process doesn't do PR, no matter how strange the quantum is.

3) humans culture and behaviour is both complex, and changes over time in a way e.g. chemistry, does not.

I think you're right, and I would go further and say that it becomes even more complicated because the higher order fields are often as old or older than the foundational ones.

We created frameworks for understanding political science and the humanities, still relevant today, millenia before any real neuroscience occurred.

It's hard to tell, for example is alchemy ancient chemistry, or something else?

It is true though that certain topics have been around so long that the study of X is itself a study.

Possibly the biggest issue is ethics. Speaking as a physicist, we can perform experiments on atoms, that would hopefully be unthinkable on humans. Even psychology gets more results if ethics are overlooked, e.g., in areas such as advertising and interrogation.

This comment really made me think, thank you for that. I agree with everything you said.

I think the danger comes from people using results that aren't (can't?) be replicated and treating them as if they were fact.

"It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so."

Well I'm extremely gratified that you even thought about it, I've been trying to express something like this in different forms with little success.

And you're certainly right that we need to take into consideration how sure we can be about a result before relying on it, while recognizing that sometimes decisions have to be made based on the imperfect information available at the time.

In my view the issue with the social and medical sciences is that they are being pulled from two opposite sides. On the one side is the motive of profit, professional advancement, etc. On the other side is the fact that science might be slow but people are suffering right now and can't wait for care. There's a similar problem with using scientific knowledge to guide public policy.

Calling it the collapse of science is definitely hyperbole, science is a method of falsification, it can't "collapse".

What is at risk is established academia and society's trust in the institution. The replication crisis is not well known outside of academic or technical circles for now. But eventually it will become a major scandal and you can't blame people for losing their trust at that point. And I think the worst part about it all is you can watch academia make the issue worse in real time. I see discussions about how to fix things but everyone ignores the gorilla in the room. They stop short of the truth, that academia is fraught with fraud and the system has set up in such a way as to promote it for anyone who actually wants to have a stable and successful career.

I've seen it myself first hand, when someone has been caught unequivocally committing research fraud they are fired and blacklisted from the community at large but are always treated as an isolated incident. The degree to which research cannot be reproduced means we have a serious fraud problem.

Academia needs to start cleaning house.

Watching a friend go through the process of attaining tenured position in Economics recently has taught me to never listen to Economists. They are all focused on impressing each other MUCH more than progressing Economics. And that's not even considering the Ergodicity crisis there.

Once you see the sausage being made, you realize "peer review" is a way to project power in academia, it is not about science.

Eric Weinstein has been talking about this lately, something he calls DISC "The Distributed Idea Suppression Complex". Some of his first hand stories are shocking, but are exactly the kind of thing those I know in academia tell me they face every day.

Reply on news.ycombinator.com