Hacker News Reader

Comment by buvanshak | Hacker News Reader

>scientists criticizing how the scientific process is currently conducted for the purposes of improving the scientific endeavor.

I think what happening here is a bit more serious. They are showing a widespread crisis. It is not just some minor feedback to improve the process.

>It does not in itself show that science is inherently untrustworthy.

I think when statistics is involved, the results are inherently untrustworthy. This is not really surprising because there is a whole bunch of ways these studies that involve statistics could go wrong. And we are still finding new ways on how this could go wrong.

Then there are things like publication bias, that takes this to a whole new level. Things like that means that a biased body of journals can project any consensus that it favors just by selecting studies that fit its narrative. The inherent issues with statistics means that you can find studies that shows any possible outcome.

View on HN

>"I think when statistics is involved, the results are inherently untrustworthy. This is not really surprising because there is a whole bunch of ways these studies that involve statistics could go wrong. And we are still finding new ways on how this could go wrong."

Another very real issue here is that malicious use of statistics can be used to show nearly anything in ways that can be extremely difficult to detect, even when the maliciousness is hidden in plain sight. And then going a step beyond that there's plain old number fudging which is almost impossible to prove since variance works as sufficient plausible deniability. And finally there is of course plain old ineptitude. Like you mention even when trying to do things completely by the book, statistics are incredibly difficult to get right.

Something that comes to mind here is the recent MIT study stating that Uber drivers earned $3.37/hour. That study was completely broken. [1] It's debatable whether the cause was maliciousness or ineptitude, but the point is that these problems arise, with a disturbing regularity, even when the most reputable of names are attached to them.

[1] - https://qz.com/1222744/mits-uber-study-couldnt-possibly-have...

Feynmann said, "The first rule is that you must not fool yourself. And you are the easiest person to fool."

There can be malice, sure. But there can also be desire to believe. And, hey, here's a statistical analysis that shows what the investigator is already biased to believe anyway...

>I think when statistics is involved, the results are inherently untrustworthy.

Ummm...are you kidding? Statistically vetted results are inherently UNCERTAIN, but how could they possibly be inherently untrustworthy?

Even if a mechanistic effect is observed, its relationship to a particular cause or influence is only established statistically. In fact, the very observation is often performed u Dee the umbrella of statistical calibration of appropriate instruments.

  As Pearson said, "Statistics is the grammar of science".

I cannot speak for the person you are responding to, but in my [agreement] of his critique of statistics, I am implicitly speaking of social statistics. I think there is a vast difference in e.g. a statistical modeling of the behavior of electrons and e.g. the statistical modeling of some sort of human behavior.

...but that just isn't true. Statistical analysis is (among other things) a way to quantify our uncertainty. If done approriately, the statistics simply communicate the role probability played in moving from the experimental premises to the results.

  The level of uncertainty in most (perhaps all) experiments involving particles in a vacuum is far lower than experiments involving human behavior. Statistics doesn't create the uncertainty, it communicates the uncertainty.

  I can use propositional logic to arrive at the conclusion that "Unicorns are real" yet I don't encounter many people throwing logic itself under the bus because of that.

People lie while speaking English every day yet I don't encounter many people claiming that English is inherently untrustworthy.

  Statistics has been a whipping boy for a long time by people who don't know much about statistics or just fail to think clearly about what is being said.

> If done approriately..

This is the whole point. It is the difficulty in doing so, or the difficulty in deciding if this was done appropriately or not is the only thing that it untrustworthy..

>People lie while speaking English every day yet I don't encounter many people claiming that English is inherently untrustworthy.

No one expect a statement to be true simply because it was uttered in English.

I'm not entirely sure what you're trying to say, and your post formatting is not helping.

I would recommend reading the article. The executive summary summarizes things. The issue is not statistics in and of themselves, but how they are used in the overwhelming number of studies - particularly those in the social sciences or related with human physiology.

Reply on news.ycombinator.com