Comment by Jabbles | Hacker News Reader

Comment by Jabbles | Hacker News Reader

Parent Full thread

The only "result" that the article could claim is the unfortunate There was no significant difference between the two teaching methods.

However, we could probably regain hope for Khan Academy by pointing out that the (five-week) summer school was almost entirely worthless for 2/3rds of the students anyway:

In terms of distribution, in each class approximately one third of the students saw some significant gains (ten percent or higher gains in percentage of questions answered correctly), whereas two thirds of the students’ scores were essentially flat (less than four percent increase or decrease).

If anything, it is this that should make us eager to explore alternative methods of teaching. I am hopeful that Khan Academy or a similar online learning provider will revolutionise education, and this article is the first step along the way to discovering how to use it.

I think you may have missed what I think is the most important point made in the article:

"It is also interesting to consider that students in the treatment group spent approximately half of the summer working on pre-algebra skills. Because the Khan software is individualized, it identified that most of our students had significant pre-algebra skill gaps and delivered instruction and practice problems to address these deficits. Students in the Khan/treatment group therefore spent up to 50% less time than the control group on the algebra content that the MDTP exam measured. The treatment group, however, still performed at a similar level the control group on the algebra measures."

That says to me that these students spent most of their time filling in gaps in their knowledge and still managed to keep up and even get ahead of the other class in a very short amount of time. They're basically saying flat out that these students learned as much of the advanced material in half the time as the other group and probably have better foundational knowledge.

No significant difference my ass.

I'm no statistical whiz but I'm not sure they can even claim There was no significant difference between the two teaching methods, but rather we have not demonstrated a signifiant difference (i.e. absence of proof vs proof of absence). Despite being supportive of the Khan Academy, I can't conclude that it isn't in fact terrible compared with traditional methods on the basis of this experiment. And given that, if this article is the first step, it's the first step in the wrong direction, like starting out to optimise software based on a faulty benchmark.

Reply on news.ycombinator.com