Comment by eganist | Hacker News Reader

Comment by eganist | Hacker News Reader

I really, really hate to ask the obvious, but doesn't this apply to damned near literally everything?

Did we genuinely need a column to remind us of this? Is this how bad it's gotten? I'm not asking rhetorically; I'm dead serious. Has publishing gone so far off the rails that we have to restate the blindingly obvious just to reboot the conversation? I'm nowhere involved in the process and will readily admit that I've got not a single paper to my name, so I'm clueless about the current state of affairs when it comes to published research.

Edit: Wow. Yeah, things actually look pretty bad if we take the opinion at its word. Particularly telling are the examples given, such as:

> For example, a breast-cancer cell line used in more than 1,000 published studies actually turned out to have been a melanoma cell line. The average biomedical research paper gets cited between 10 and 20 times in 5 years, and as many as one-third of all cell lines used in research are thought to be contaminated, so the arithmetic is easy enough to do: by one estimate, 10,000 published papers a year cite work based on contaminated cancer cell lines. Metastasis has spread to the cancer literature.

It indeed seems to be bad. If you try reading some papers you can find everything from conclusions obviously based on formal fallacies, confusing statistical significance with practical significance, experiment designs that couldn't provide any new information no matter how they turn out, to outright bizarre flaws of logic such as dietary intakes calculated so that the body isn't able to excrete more than it absorbs.

The medical science has decayed into a strange cargo cult where experiments are made and papers are written, but the essence of science, the pursuit of better understanding, is lacking. It's no wonder there has been no major progress in the last few decades and we have an epidemic of obesity and other metabolic disorders.

> but the essence of science, the pursuit of better understanding, is lacking

When I read articles and books from scholars from the 19th and beginning 20th century, I'm often surprised how broad their horizons were and how cautious in their thinking. It's always a pleasure to read.

Where can I read about medical science having been more focused on understanding in the past? I think the focus on small increments in (relatively) unimportant medical knowledge is perhaps the greatest tragedy of our time.

Also, who calculates dietary intake that way?

"Confusing statistical significance with practical significance" -- that's a great phrase!

>"Where can I read about medical science having been more focused on understanding in the past?"

As far as I know no one has really written this up, it is just something you figure out by reading the literature. You see that despite the technological limitations at the time they tried to include tables of the data in as much detail as was practical and rule out competing explanations, etc. Over the decades this degenerated into testing null hypotheses and reporting p-values and averages. Now there is somewhat a recovery due to the sheer ease of transmitting information, but it is still amazing how little this is taken advantage of in general.

I've thought about a book but not sure people would buy it. How much would you pay for something like that?

Certainly it does -- and yes we do. Things are so bad that entire fields of science are best disregarded until a few good meta-analyses come out. Or, better yet, large randomized trials with predetermined criteria for success or failure, and no fishing allowed.

(Example: remember that "10,000 hours to expertise" thing that Malcolm Gladwell was pushing a few years ago? Forget about it; the whole thing failed to replicate. And forget about Malcolm Gladwell in general, since he makes a career of this sort of borderline-fraud.)

Your criticism of Gladwell is oddly unspecific given the nature of your criticism. You fail to state what exactly you criticize in his 10,000 hour statement.

When I read the kind of criticism you seem to be referring to, like this one:

http://www.businessinsider.com/new-study-destroys-malcolm-gl...

I notice that the examples are silly: They equate "success" with "mastery". For example, they write

> Then there's a band like the Sex Pistols, who took the world by storm even though Sid Vicious could barely play his bass.

So they themselves say he "could barely play his bass"! How does that refute the mastery theory? The Richard Branson example is about "success" and not "mastery" too - I bet Richard DID have "10,000 hours" of business practice.

Not that Gladwell made any specific predictions either AFAIR. I wish the people criticizing his work would made an example of how to be better - but it seems the criticism itself sets new low points in quality.

"What Malcolm Gladwell REALLY Said About The 10,000 Hour Rule":

http://problogservice.com/2012/03/15/what-malcolm-gladwell-r...

> “Malcolm Gladwell said you have to have 10000 hours in a subject to be an expert,” they will often state. The problem is, they’re repeating a misquote from someone else who has never read the book.

Also note how Gladwell quotes scientists - so any criticism should be directed at those:

> “The emerging picture from such studies is that ten thousand hours of practice is required to achieve the level of mastery associated with being a world-class expert—in anything,” writes the neurologist Daniel Levitin. — p. 40

Oh, and even the stuff that was true was completely misinterpreted. For example, he cited a study of students as showing there was no "math talent", only diligence. The setup actually was quite interesting: there was an additional informational questionnaire that wasn't about skills at all, and there was a correlation with conscientiously filling out that questionnaire and scoring high on the actual skills test.

However.

The effect only appeared in 2 grade levels and only in some years, so extremely weak evidence. But that's only a small detail. The big one was that the effect was only evident at the national level. So yes, there are no nations that are more talented at math than others. In other news, water still wet. However, the data, even if the flimsy bits are taken at face value, simply do not show at all that there is no math talent at the level of individuals, and in fact showed the exact opposite.

Details.

Reply on news.ycombinator.com