Yeah, I've seen this xkcd. It's on point as usual, but it doesn't show the frightening thing: it's the scientists who are going "Whoa" and believing in the results they've produced...
250 karma · joined December 6, 2013
People there are re-doing the same experiment over and over until it gives them the result they want, and then they publish that. It's the only field where I've heard people saying "Oh, yeah, my experiment failed, I have to do it again". What does it even mean that an experiment failed? It did exactly what it was supposed to: it gave you data. It didn't fit your expectations? Good, now you have a tool to refine your expectations. But instead, we see PhD students and post-doc working 70h hours week on experiments with seemingly random results until the randomness goes their way.
A lot of them have no clue about statistical treatment of data, making a proper model to try and test assumptions against reality. Since they deal with insanely complicated system, with hidden variables all over the place, a proper statistical analysis would be the minimum expected to be able to extract any information from the data, but no matter, once you have a good looking figure, you're done. In cellular/molecular biology, nobody cares about what a p-value is, so as long as Excel tells you it's <0.05, you're golden.
The scientific process has been forgotten in biology. Right now it's basically what alchemy was to chemistry.
I very happy to see efforts like this one. Sure, they might show that a lot of "key" papers are very wrong, but that's not the crux of it. If there is a reason for biologists to make sure that their results are real, they might try to put a little more effort into checking their work. And when they figure out how much of it is bullshit, they might even try to slow down a little on the publications and go back to the basics for a little while.
I'm sorry about this rant, but I've been driven away from a career in virology by those same issues, despite my love for the discipline, so I'm a bit bitter.
We now have ways of obtaining mountains of data (RNA-chIP comes to mind, but high-speed sequencing is a big one as well), but we don't know how to process it. We end up with thousands of candidate proteins, but then someone has to go through them one by one. Things are incredibly primitive. The huge pain point coming up is our ability to process information, which is very limited. How do you deal with machine generating TBs of data/day ?
I think what's lacking is a culture of automation. Physicists do it, chemists do it, but biologists still do everything "by hand", and not only is that imprecise, is also slow and costly. We also need better ways of obtaining information about other's work. When there are 1000 paper a year published on your subject, how do you filter ? How do you even find the time to read papers in other subjects, which could be interesting in their methods, or just for general intuition and knowledge ?
What we need is good algorithms, able to replace curate our sources of information well, and machines able to do the bulk-processing now needed in biology. Some things are coming up (I'm co-founding a start-up to do exactly this), but it should have happened 10 years ago.