Hacker News Reader

Comment by danieltillett | Hacker News Reader

What we really need to find a way to support long-term, high risk projects driven by young scientists with breakthrough ideas. At the moment all scientists have to live from grant to grant which makes it near impossible to go after the really risky projects. If you try a difficult project and fail then you are out of a job.

We also, because of the wonderful system of peer review, pretty much only give grants to old established researchers with long track records. These scientists are great at running mini-empires, but not so good at coming up with the really novel ideas that we so need.

View on HN

> What we really need to find a way to support long-term, high risk projects driven by young scientists with breakthrough ideas.

Fellowships typically fill the roll of paying for long-term, high risk research.

If it needs to be weighted away from established researchers, make it a lottery.

Current funding models are optimised for convergence on leading hypotheses. A lottery would increase coverage.

Having a lottery is a good idea and one I support for grants in general. The grant review process should be simply "is this idea viable and worthy of funding - yes or no”. If yes put it into the lottery and fund as many good grants as you have resources. Peer review works well for this type of sorting, but it is terrible for trying to pick a top 10% idea from a top 15% idea. It just isn’t possible for even the best scientists to be able to do this consistently.

The problem with fellowships is that the longest I know of only go for 5 years and most of the junior ones are only for 3 years. This is just not long enough to take the chance on failure.

If you want to encourage young talented people to tackle really hard and high risk problems you need to provide funding for at least 10 years.

I would go as far as adding a restriction that recipients of these high risk fellowships can’t publish in anything but the absolutely top tier journals (Nature, Science, Cell, etc) for the first 7 years so they don’t get distracted from the high risk work that other scientists can’t afford to do.

In my opinion we are missing out on great discoveries because we allow a small number of humans determine if a scientific research question is worthy of funding. If a single scientist thinks an idea is worth doing and can find the resources to support it, she should do it.

We will move to a system where we only ask 'can this question be answered by the scientific method?' If we use the scientific method to answer these questions we will uncover secrets of the universe that would never survive in the current system humans have designed.

The best scientists do not design their experiments with a goal of publishing in top tier journals. And as a community, we should not be encouraging this behavior.

The best scientists investigate questions out of pure curiosity and as a side effect publish the most impactful research.

Young talented people are already tackling really hard and high risk problems. If this is truly what young scientists want to do, they will find a way. 10 years of guaranteed salary is not the solution, but giving everyone the opportunity to be a scientist may be.

>Young talented people are already tackling really hard and high risk problems. If this is truly what young scientists want to do, they will find a way. 10 years of guaranteed salary is not the solution, but giving everyone the opportunity to be a scientist may be.

I love your idealism, but having climbed the greasy pole to academic tenure (I have since left science) I can say that you really can’t afford to work on the high risk projects. When you are a junior scientist the more senior scientists control what you can work on and they want you generating publications. When you are a post doc you can’t afford to not get publications out and hence you have to work in areas which will produce results within 2 years. Once you are a senior scientist you are forced to work on projects that will generate x number of papers within the 3 year grant period or else you wont get any further funding. At every stage of your career you are effectively forced to work on projects that are guaranteed to generate publishable results within the next 2 years.

The reason I suggested that the scientists awarded one of these 10 year fellowships are not allowed to publish in anything but the top journals is to remove the pressure of publishing off the fellows and allow them to concentrate on solving the difficult problems. The only reason I suggested an exception for the top tier journals is in rare case the fellow made a lucky breakthrough - we don’t want them sitting on some important result for years just because they not allowed to publish.

I have many high risk/high reward projects that I would have loved to have worked on when I was a professional scientist, but I could not afford to do so. I knew if I failed to produce consistent publishable results I would be out of a job. Even once I had tenure I could not in all honesty ask my students or post docs to work on projects with a significant risk of failure. We have created a system where we have a high probability of making incremental progress, but almost no chance of big breakthroughs.

I've always wanted to run an experiment where NIH or NSF split the proposals randomly 50/50. Group A gets peer reviewed and awarded the way the current system works. Group B gets peer reviewed and the top half of proposals gets put into a lottery and awarded at random. I am willing to bet the outcomes and impact of Group B will always be better than Group A.

Cindy I suspect you might be right - especially if we measure for breakthrough discoveries.

The problem you face running such a trial is those scientists that control the current system are not too interested in trying anything that might take away their control. They love to mis-quote Churchill and claim their is no better way than the current system, but they are none too keen to try any alternatives.

I'm currently working through preparing for a candidacy exam, and I've found a number of errors on a Physical Review article...yes, a Physical Review. They aren't the crux of the paper, but they are fundamental equations that they apparently implemented in their code (of course, I don't have access to that code), and these updates are the crux of the paper. If the equations they wrote are wrong, how can I trust their results?

Recently a senior researcher admitted to me that from a journal he referees for, he was referee-ing for a paper and found 10 issues with it. However, the editor came back to him and told him accept it.

You're damn right it's stacked, and sometimes "long track records" just means cosy with the publisher. It's not like there isn't corruption elsewhere in the world, but young scientists do not have start-ups. They do not have opportunities or the means to go against the grain and get into those risky projects unless it's after the decade or two of getting cosy with others in the academe and building those mini-empires. You're very right they need a venue, an accelerator for grad students, I suppose.

Just curious, have you asked the researchers for their code? I've found that some people are willing to share their code and others aren't (some are just shy and embarrassed about how ugly it is, others have less justifiable reasons like it gives them a competitive advantage) Have you discussed your findings with them? Have you considered writing a comment outlining their errors (you might talk to your adviser about that)?

Refereeing is difficult to do well. On the one hand,in a Physical Review Letter, the article should be more broadly accessible, but on the other hand, it needs to be technically correct. I remember once reviewing a methods paper (different journal) and checking each equation and each integral (and finding a few minor errors) because it wasn't likely to be checked so much once it made it into a black box--but that took a lot of time! For myself, I've never had an editor tell me to "accept" a paper with errors. I have given a report and had the editor make a decision the other way, but that's different then asking someone to amend a report.

As for taking risk, I think the best advice I can offer is to have a portfolio of projects. I'm more senior in my career, but I did the same thing as a graduate student. Some things I work on have little chance of success, others more. This will vary by field and how expensive your needs are--but part of choosing an adviser (especially at the postdoc level) is choosing one that will allow you to explore your own ideas. I feel that my duty to postdocs is to offer some ideas that may work. If they have other ideas that they'd like to explore, then I view my job as to offer my expertise to support them...Micromanaging is too much work!!!!!!!!!!!

Thank you for the advice. I appreciate it.

>Recently a senior researcher admitted to me that from a journal he referees for, he was referee-ing for a paper and found 10 issues with it. However, the editor came back to him and told him accept it.

I have had this happen. I have rejected papers as fundamentally flawed and the editor has just published the papers unchanged. It make you rather cynical about the whole peer review process.

> These scientists are great at running mini-empires, but not so good at coming up with the really novel ideas that we so need.

Sounds like some IT departments I know.

Reply on news.ycombinator.com