Problems in Education | Hacker News Reader

Problems in Education

lesswrong.com

I'm not sure this collection of anecdotes will be news to someone who has been following education policy for a while but I do think the topic is important. With the importance of the topic in mind, I'll suggest here some better things to read on improving education.

http://stuff.mit.edu:8001/afs/athena/course/6/6.969/OldFiles...

(review of a very important book that essentially every parent in the English-speaking world ought to read)

http://hanushek.stanford.edu/publications/teacher-quality

(a collection of articles on teacher quality by an experienced education policy researcher)

http://educationnext.org/

(website of a journal on education policy issues)

For what it's worth Hanushek is most famous for his theory that a student having a teacher in the 85th percentile or better for test score improvement for five years in a row would be enough to close the gap between low-SES and high-SES kids, a theory which has been soundly disproven. I have no idea about the quality of his other research, but I'd tread with extreme caution there.

edit: C.f. The Death and Life of the Great American School System by Diane Ravitch, the paperback edition, esp. chapters 6 - 9 and the epilogue.

I'd have to look up the reference you kindly shared, but I sure hope Ravitch had help with her mathematics in that analysis, as she is the person who doubts the general usefulness of even beginning algebra as a high school graduation requirement. (Her higher education was in history, not any quantitative subject as Hanushek's was.)

She is citing other studies that attack Hanushek's analysis, but she isn't citing any mathematical arguments that she created herself. There are a number of problems with Hanushek's argument, among them:

- The ratings are completely unstable, so a teacher who is in the bottom 20% one year has a pretty good chance of being in the top 20% the next year.

- Hanushek doesn't account for forgetfulness when evaluating the differences between teachers. So if you retest the students on the same material two years later, it turns out that the difference between a great teacher and a mediacre teacher is only about 1 / 8th as much as Hanushek had supposed. And the teachers that do the best in the short term (at the end of the school year) have little correlation with the teachers that are actually the most effective in terms of getting kids to know the most after two years, which is what is actually important.

- Even if the teacher ratings were stable, which they aren't, they're not on an interval scale, meaning that the comparisons between different teachers are largely meaningless.

- Students aren't assigned randomly to classrooms, and the differences between their abilities and backgrounds can't fully be corrected for for various reasons. (One of which being that again the tests are not on an interval scale.)

- In order for the VAM to become at least fairly stable it takes about 7 years, by which point about 60% of teachers have already been fired or quit anyway.

- VAM only works for math and literacy and not the other subjects that aren't tested.

- The VAM ratings are extremely test specific, and if you give a different test of the exact same material to the same kids then the teachers that are 'effective' or 'ineffective' are often wildly different.

- VAM only works if you have data for kids from the previous year. However the data tends to be missing for about a third of kids, especially in low-SES communities where people move around much more, which means that VAM is much less accurate in minority communities due to having less data to draw from and therefor less statistical power. Which means that it's effectively firing/disciplining/rewarding teachers at random in these communities, and therefor discouraging good teachers from teaching there.

There are other issues too, I'm happy to email you my notes if you'd like, which are mostly from the sources that Ravitch cites and not from Ravitch herself. (The sources are mostly things like Department of Education studies.)

"It's pretty hard to argue with the data."

Would be nice if there were any actual data in this article.

Last paragraph from the article:

> I can provide citations for any statements I've made, but since nearly everything I said is NDA'd I'd like to be careful to anonymize my citations, making sure to find national studies instead of state or county specific studies.

So I believe that if you are interested in any particular claim the OP made, you can contact him for some resources.

This article has a pretty bad smell. The author makes a lot of fairly extreme claims but these are mostly based on his own anecdotes or non public data sets. The claims that

"you can separate all advanced math teachers easily into two categories: Okay with blacks in their classroom [...] Not okay with blacks in their classroom. Whites end up succeeding, blacks end up failing"

and

"the evidence shows that teacher recommendations have zero correlation with aptitude in a field"

are really quite dubious and almost certainly false. How did they measure whether they were "okay with blacks in their classroom"? As many others have said in the original thread, this is probably a hoax.

Actually those claims are basically true. If you look at the graph from my blog post here:

http://alexkrupp.typepad.com/sensemaking/2009/02/the-most-im...

What you see is that the data points from all three ability groups are basically overlapping, meaning that the students get sorted into ability groups based on factors like race rather than on actual ability.

That being said, while you might be able to model the bias of teachers and school systems by using an algorithm, I've never heard of schools actually using an algorithm to explicitly sort students based on race, which he implies that they are doing.

However, assuming he doesn't actually mean this, I think the post is actually legit but just poorly written, perhaps purposely to promote his startup.

The big assumption that everyone tends to make when talking about public education is that the politicians, school boards, principals, and teachers are advocates for students.

Sure, some of them are, but the vast majority of them are looking out for themselves. (I don't blame then, the system pushes them to this) They're making decisions on what will be best for their situation, and they are not really held directly accountable to those they serve, the students.

If anyone has tried to fight the system (like the author), they quickly realize this. And the only effective way I've seen to balance the power is school choice vouchers. Otherwise you're condemned by your zipcode unless you're wealthy enough to afford private school tuition.

I never even heard of a system that automatically selects students for any classes. You choose what you want to do, and you can go do it. At least until you go to university or something.

And I thought our school system was bad...

This is why evaluation is so important.

Also, I think there is an analogy in how startups talk about number of users, instead of dollars earned.

People need to use the correct metrics for success.

Almost all of this article, importantly starting with the "background" material in the first paragraph, is wrong. It is so wrong that I wonder: is it intended as a joke?

Care to elaborate on what is wrong about it? I don't have a lot of knowledge about the inner workings of the public education system (outside of going to one for 12 years), but from my observations as an outsider, the article seemed plausible enough. So, I am interested in anything you can dispute.

Sure. Just for a start, sentence 3 is wrong.

This is not an issue in the UK. In our schools teachers are provided with data based on prior achievement and demographics of all their students. If a cohort of students fail to reach their expected target then the teacher is usually considered responsible. Of course, rigorous adoption of a target based system also leads to teachers, schools and exam boards gaming the system and subsequent grade inflation, but that is a different problem entirely.

"If a cohort of students fail to reach their expected target then the teacher is usually considered responsible."

Or at least the alarm bells start ringing and there is a conversation (UK based myself). A 'cohort' is the word I usually use for a whole set of classes, e.g. a year group. More than one teacher is usually involved. We also have independent information on teaching through QA observations and OFSTED. Your picture is fundamentally accurate but it is a tad more complex in my experience.

Reply on news.ycombinator.com