Controversies like these make me glad I work in the field of programming...while arguments over what algorithm or frameworks work best are less sexy than what the sciences cover, at least with narrow claims, it's easier to concretely argue for one or the other...and then of course, if we're talking about open-source, then the errors of ambiguity are even scarcer.

It seems like a large part of the problem, at least in these psychological experiments, is that there isn't a structured/uniform way to describe preconditions, methodology, assumptions, and measurement practices...OK, so you used 40 undergraduate students for your original experiment...how did you pick them out? How were their characteristics accounted for (e.g. age/gender/major/health/etc)? Did any of them ever see the Trainspotting scene before? Have any of them ever seen a movie of equal grossness? How many minutes did you wait after showing them the Trainspotting scene? In what order did you hand out the questionnaire? How did you seat the students? etc. etc. etc. etc.

A totally honest researcher might have trouble enumerating and enforcing all of the different controls, never mind communicating them to other researchers. From the research papers I've read, the ability to communicate these facts doesn't seem to be of uniform quality.