For context: https://statmodeling.stat.columbia.edu/?s=brian+wansink
(he's a Cornell prof with dubious research/PR tactics)
(he's a Cornell prof with dubious research/PR tactics)
> P-hacking shouldn’t be confused with deep data dives – with figuring out why our results don’t look as perfect as we want. With field studies, hypotheses usually don’t “come out” on the first data run. But instead of dropping the study, a person contributes more to science by figuring out when the hypo worked and when it didn’t. This is Plan B. Perhaps your hypo worked during lunches but not dinners, or with small groups but not large groups. You don’t change your hypothesis, but you figure out where it worked and where it didn’t.
This is horrifically bad research practice and basically guarantees that all of your results will be completely p-hacked.
https://statmodeling.stat.columbia.edu/2018/09/23/tweeking-b... is another hilarious example.
It's totally fair to do subgroup analysis, etc to try to better understand the data, but you have to keep in mind that you can't claim statistical significance on those without very careful control over researcher degrees of freedom and other related issues.
So every study "discovers" something new, but it's not because of biases and false positives.