The standard to graduate isn’t uniform, so using graduation rates biases towards states with weak requirements.

> Obviously that's meant to act as a proxy for learning

That is the INTENT, sure. But does it work? If you have someone that graduated from a school A and someone that failed to graduate from school B that is in an entirely different area, can you draw any meaningful conclusions? Particularly knowing that school A has no particular requirements on why you graduate, while school B does?

I'm far more likely to complain about "But when we disaggregate student performance scores by racial categories (white, black, Hispanic, and Asian), the rankings change dramatically." I wouldn't want to go too far with that before ensuring that I wasn't trying to curb swimming deaths by keeping Nick Cage out of movies: http://tylervigen.com/view_correlation?id=359

Right, surely these states have some sort of standard that must be met for graduation even if it is not passing some exam.

First a point about bias. They are quick to point out the bias of traditional scores. But they do not point out that their bias is for libertarian causes. Had they not found that more libertarian states perform well by their measure, they would never have published this.

Removing graduation rates is definitely troublesome. Any school system could then improve its numbers by encouraging the worst students to drop out. Problem students both take more resources to educate and produce worse scores. That's bad for efficiency, but it is not obviously bad for society.

I am also troubled that they are using measures of performance in grades 4 and 8, but not something like SAT scores. I understand that SAT scores could be hard to get. But a lot of what people care about in an education system is how well prepared you are for college, and not how far along you were in grade 8.