Hacker News Reader

Comment by a9h74j | Hacker News Reader

> As is clear from the empirical evidence reviewed in the previous sections, the likelihood that cognitive training provides broad cognitive and academic benefits is very low indeed; therefore, resources should be devoted to other scientific questions—it is not rational to invest considerable sums of money on a scientific question that has been essentially answered by the negative. In a recent article, Green et al. (2019) took the exact opposite of this decision—they strongly recommended that funding agencies should increase funding for cognitive training. This obviously calls for comments.

I wonder how the lack of direct results relate to cognitive rehabilitation for persons with some medical etc issue. Could one value be to make it very evident to a person that they do have a deficit and need to work on compensation strategies? So measured by overall well-being cognitive rehabilitation could produce positive measurable outcomes?

View on HN

There's a problem I have with this kind of science.

It boils down to "We tried to achieve X by doing Y and failed, therefore X isn't possible".

Assuming Y is the correct way to do X and then giving up. Instead it should almost be something like a competition. "We tried to achieve X by doing Y_{1..n} and these n had statistically significant effects of these sizes"

When you have a lot of people trying to accomplish something, some of them will end up being successful if the thing is possible, but there is an art to finding that optimum and published science really often seems to fall short.

We also have Q P S R T U V and W that need funding and aren't getting it because X is getting all of it.

If X is too hard try something else that's easier.

I think you are mischaracterizing this meta-analysis. First, a meta-analysis is never about just trying one thing; that's what a single study is for. A meta-analysis looks at many studies, over a long period of time and tries to find consistent patterns that emerge from the whole. That's how we try to systematically figure out if results are by chance, or if there's a real effect. The pattern these authors are pulling from all of these studies is there is no real effect. When you find no real effect over many studies for a long time, you inevitably have to confront the question: is what we're trying to do possible?

> When you find no real effect over many studies for a long time, you inevitably have to confront the question: is what we're trying to do possible?

Devil's advocate: The same would apply to say, Alzheimer's treatments, where we've had no real effect over many studies for a long time. Should we give up?

Lack of progress is a good reason to backtrack up the search tree. Which is exactly what we are seeing with alzheimer’s research. Amyloid hypothesis isn’t working, so backtrack. It’s a logical error to extend that to “give up”

The logical extension of this would be that perhaps it's time to not spend as much effort on some approaches at cognitive training, but not to give up on possibility of the entire idea.

>When you find no real effect over many studies for a long time, you inevitably have to confront the question: is what we're trying to do possible?

This is what I'm criticizing. The methodology for searching for real effects is broken.

one problem with meta analyses is that they often collapse studies accross theoretically important details, and thus unsurprisingly find weak or inconsistent effects. I saw this in a lot of metaanalyses of memory function.

Reply on news.ycombinator.com