Comment by Fomite | Hacker News Reader

Comment by Fomite | Hacker News Reader

Parent Full thread

Am a researcher, and also work in data governance, data sharing, etc. for medical research data.

It's not about an "open source mentality" - almost all of my analytical code is online, and when I work with simulated data sets (the bulk of my work) it's freely available.

It's about incentives, and effort. A couple examples:

1) Primary data collection is hugely time and resource intensive. It's hard to be okay with all of that, getting a single paper out, and then immediately having that be expected to be freely open and available.

The argument that it's necessary for reproducibility is a compelling one, but also one that's much rare than what most people are looking for, which is to do new and novel research using your primary data.

2) Research has a potentially long time from data collection to publication. It's entirely possibly the reason isn't "I may want to work on that someday" but "A graduate student is working on that right now". I have been in the position of potentially having a graduate student be scooped on something, and it is terrible (it ended up being a false alarm). They're potentially also the person who put in a lot of the labor on publishing the paper in the first place.

3) Sharing comes with an unknown expectation of support, documentation, etc. wherein there's absolutely no incentive structure in place to support the work that takes. I write in time and budget to do so in my grants, but that also mean my grants have less science in them than other people's, and I have to rely on that mattering to someone enough to make up the difference.

4) Genuine data ownership issues. Health data is especially complex in this space. Do you want a simulated data set that will give you the same broad patterns? That's easy enough. But if you want the real data - that's potentially a very complex issue wherein even if I say "Sure, let's get that process started..." you're potentially looking at a year or more before you get the data.

This x100.

I'd also add that primary data collection is risky. You can spend months to years working to get data, only to end up with nothing---not even a null result---if the experiment itself goes south. I lost over a year of my PhD when an animal I had spent ages training fell while playing. A classmate had a virus wipe out her colony of genetically-engineered mice. Using someone else's data obviates these sorts of risks--while doing a better job preparing you for non-academic careers too!

As a result, I think it's not totally nuts for the original data collectors to maybe get some kind of "risk premium." I can imagine a lot of ways to do it, but early/exclusive access to data has been one of them.

A colleague of mine during graduate school who by consensus did everything right, and then just...no one in her cohort had the exposure she was interested in. To this day, no one is sure why.

The answer for a lot of this is to put in the hard work of building relationships with people who collect a lot of primary data, and make sure your work amplifies their work in a way that benefits you both. But that's a lot of effort, hard, and uncertain in its own ways.

I don't have any direct research but am tangentially aware of some of this. I guess my question comes down to why getting scooped matters?

I definitely understand the worry when it comes to high profile discoveries, but surely the work a graduate student is working on, isn't usually that high profile?

And to be clear, I'm not downplaying the importance of anyones contributions, but rather why the focus isn't shifted onto where data is sourced from? If someone used your data, shouldn't it be immediately obvious if they also have data publishing requirements? And if thats true, why isn't it a bigger faux pas to use another labs data before that lab has even published their paper on it?

In some ways the graduate student is working on the most important work of their career. A PhD requires novel publications and if you get scooped on the main result of your thesis so that you don’t end up with the published paper on that topic very few organizations will grant the PhD. So this can result in losing out on getting a degree you put five years of your life into.

Being first is almost everything -- the product in academic research is new knowledge.

How much press was given to the second time the LHC discovered the Higgs, with higher precision?

Robert Dicke, in the running for "greatest experimental physicist in history", was working rapidly toward discovering the cosmic microwave background, with a keen physical insights driving the research. His team was beaten by a team at Bell Labs, who, when they discovered an irreducible isotropic source of noise emanating from the Universe, literally called Dicke [1] to help them understand the signal.

Who got the Nobel Prize? The team from Bell Labs. Never mind that Dicke literally wrote the companion paper explaining why the Bell Labs measurement was so important. Why? Because they were first.

Regarding graduate students: It is extremely common in physics to have the lead author (and coordinator) on important/cutting-edge research be a graduate student. It is only the largest collaborations where this isn't always the case.

Graduate students tend to follow a single thread of research from beginning to end, learning the ropes on the way, while faculty generally manage and mentor multiple threads (each with a student) simultaneously. When it comes time to publish, frequently it is the student who knows all of the details of the measurement the best.

For your final question, the answer goes back to "because being first is almost everything", combined with "we have to give the credit to somebody, right?". If someone notices that the eighth word on 50% of Guinness World Record citations is "person", should the credit for the discovery go to the person who noticed that interesting fact, or should it primarily go to Guinness World Records, who assembled and produced the dataset over decades of work?

Or, more-concretely, when someone applies for and receives fifteen minutes of time on the Hubble Telescope and, using a clever time-domain analysis, happens to find a repeating optical signal that blinks 13 times, then 53 times, then 13 times, then 53 times (as good a SETI candidate as you'll find), who gets the credit? Is it the researcher with the keen insight and good luck, or is it the huge amalgamation of humanity, spanning a half-century of science and engineering, that made such a measurement possible?

[1] https://theconversation.com/the-cmb-how-an-accidental-discov....

Graduate students have like four good years of research time. If your work is scooped it will almost guarantee low citation counts but very likely it means that the work cannot be published at all. A year down the drain.

If you want a faculty position after graduation you've got one shot to create a strong research profile. Getting scooped once can close career doors forever.

People also often don't understand what grad students do. They do all of the research. 100% of it. PIs are funding the lab and providing mentorship. The first authors on major groundbreaking papers are usually grad students.

So, the caveat to all of this is I'm actually in a field (epidemiology) where getting scooped matters less in many cases than it does for some others (math).

Getting scooped doesn't just matter for high profile discoveries. The example I was thinking of is the graduate student's research asking "Does this thing the field is doing make sense?"

That's a Yes or No answer. There's not a lot of interest once the question has been answered. Even if it can be published, it'll be a harder road, potentially finding itself in a less prestigious journal, etc. At worst, it's a write-off. Most graduate students are coming out with 2 or 3 paper dissertations, so the loss of one, or taking a hit on another one, can set someone behind, dramatically change the tenor of their job search, etc.

"And to be clear, I'm not downplaying the importance of anyones contributions, but rather why the focus isn't shifted onto where data is sourced from? If someone used your data, shouldn't it be immediately obvious if they also have data publishing requirements? And if thats true, why isn't it a bigger faux pas to use another labs data before that lab has even published their paper on it?"

I mean, it might not be the only thing that's been published on that data - that's the point. Once it's publicly available, it's publicly available, even if people are still working on it.

Indeed, one of the "Data is available upon reasonable request" things is "We have a student working on this, could you not?" (another being "You're a crank, and we're not sending you this to misinterpret/misuse it"), but a lot of open data folks actively dislike that for obvious reasons.

Reply on news.ycombinator.com