Hacker News Reader

Comment by lstodd | Hacker News Reader

> It is done on purpose.

The sad thing is that it isn't. It's just how the things are done.

src: friends in HEP

I used to work (current PhD student) in HPC (switched specialties) and I was extremely surprised that not only did these people not share code (DOE actually encourages open souring code), but they would not even do so upon request. Several times I had to get my advisor to ask the author's advisor. Several times I found out why I couldn't replicate another person's code. It is amazing to me that anyone in CS does not share code. It is trivial to do and we're all using git in some form or another anyways.

Sharing source code for a HEP experiment is not that easy or possible at sometimes. A lot of stuff is done by different group and whole framework used and overall a lot of raw data collected and reconstructed involved. To analyze those even if available you willl need a lot of people with a lot of resources. So even it made available (which would be a lot of efforts itself) it wouldn't make much sense for replication purpose.

How is it possible to peer review a paper when the only people qualified to do so are the one's involved in the research. Seems like a massive issue. Don't want to cast to much shade here, but there's a fair number of people that called out high energy physics for having problematic methodology. See Constructing quarks by Andrew Pickering. Ultimately, it should be CERNS job to release the data, yes even if it is terabytes in size, because that's the whole point of science.

>How is it possible to peer review a paper when the only people qualified to do so are the one's involved in the research. Seems like a massive issue.

Peer review isn't what it's chalked up to be. Basically it's two other people in the same field saying "There's no blatantly obvious issues with this paper" and even that isn't always guaranteed. Reviewers don't make any efforts to actually replicate the research, crunch the data, etc.

So what? Tough shit! It's still simply the only way to replicate or audit something.

If it's hard or tedious... well so what? So is life. That's most of science is exactly that rigor.

You might as well say no one else can grow a potato because of all the work your farm had to do all year to arrive at your crop. Yes, any other farmer will have to do all the same stuff. It's an utterly unremarkable observation.

It's just hard to do. But there are efforts in this direction e.g. https://analysispreservation.cern.ch/

src: I used to be in HEP

Reply on news.ycombinator.com