Hacker News Reader

Comment by delichon | Hacker News Reader

Approximate replication rates in psychology:

  social      37%
  cognitive   42%
  personality 55%
  clinical    44%

So a list of famous psychology experiments that do replicate may be shorter.

https://www.nature.com/articles/nature.2015.18248

View on HN

I think one would wish the famous ones to be more often replicable.

There may be minute details like having a confident frame of reference for the confidence tests. Cultures, even psychologies might swing certain ideas and their compulsions.

Nonreplicable publications are cited more than replicable ones (2021)

> We use publicly available data to show that published papers in top psychology, economics, and general interest journals that fail to replicate are cited more than those that replicate. This difference in citation does not change after the publication of the failure to replicate. Only 12% of postreplication citations of nonreplicable findings acknowledge the replication failure.

https://www.science.org/doi/10.1126/sciadv.abd1705

Press release: https://rady.ucsd.edu/why/news/2021/05-21-a-new-replication-...

This feels like some sort of truth telling paradox, where if you assume the study is true, then seeing a citation like this means it's likely not true.

This is at least partially a failure in publication. Once a paper is published, it's usually left up in the same state forever. If it fails to replicate, that data is published somewhere else. So when someone references the paper, and the diligent reader follows up and reads the reference, it looks convincing, just as it did when first published. It's not reasonable to expect the reader, or even the writer, to be so well versed in all the thousands and thousands of papers published that they know when something has failed to be replicated.

What we need is for every paper to be published alongside a stats card that is kept up to date. How many times it's been cited, how many times people tried to replicate it, and how many times they failed.

Isn't the unexpected more famous than the expected?

Thanks for providing the reference, that's useful context. Those are awful replication rates, worse than a coin flip. Sounds like the OP can add their own introduction to their list. From the introduction:

> Most results in the field do actually replicate and are robust[citation needed]

Reply on news.ycombinator.com