Hacker News Reader

Test are a consequence of a very real problem of evaluating teachers. Since neither parents nor children have in most public schools any choice over the school and teachers in it, regular consumer-driven quality controls that work on most markets are absent. However, quality problem is still real and existing - there are, obviously, good schools and bad schools, good teachers and bad teachers. Assume you are a well-intentioned educational administrator, and you want to improve the quality of education. How you do that? How you know which school is doing good and which is doing bad? How you know which teacher is beneficial and which is not? How you know which policy is beneficial and which is harmful? And most of all - how you do it in a way that is scalable to state and national levels and can be centralized and processed at huge scales (without which Department of Education has no data on which it can base its policies)? Obviously, standardized tests look like a perfect fit for this problem. So there's no mystery in this, it is a natural result.

Of course, there are a lot of deficiencies in this approach, which you can see and teachers can see it too. But given centralized, state-driven education system, what else you can do? You can not have absence of quality controls - this would have immense corrupting potential and the system would have incentive to deteriorate very quickly. You can not have individual approach - DoE can't account for each school in each small town, they just don't have resources for that. So with current approach, testing is inevitable.

While I appreciate that you have decided to test the teachers, and that you have unions which come in the way of it, I am greatly amused by the ill conceived plan to achieve your goal.

Its remarkable really. Years, decades of seeing kids die to suicide and cram pressure and this is what you choose.

So that you can test your teachers?

Anyway, your test doesn't test teachers, it tests students.

And then you assume it is a good proxy for teachers.

So just to be clear.

1) Tests are a perfect proxy for taking that test -> 2) Then they are a proxy for test taking ability in the student 3) Then a proxy for student understanding of the subject 4) And thence a proxy for teacher ability.

I'd understand if we were doing chemical analysis with 4 steps involved in the process.

But with human beings? Kids? The fundamental chaotic units of society?

I really have to point out that this is logic that gets most startups killed, or logic on which they pivot to something smarter.

I know you aren't the one responsible, but I really can't help but be perplexed. Once upon a time you guys were known as the place where "stuff just works."

Your older education system did a great job. Sure it may not have made everyone a rock star but frankly - not everyone is cut out to do so at that juncture in their life.

If you say that there is nothing else that can be done, then I accept and understand. But it is a terrible terrible outcome.

I didn't decide anything. I just explain that the decision to test is inevitable.

>> Years, decades of seeing kids die to suicide

Oh come on. You could at least wait couple of lines before you drop "you are a child killer" bomb. This is complete bullshit. There are very unfortunate cases of child suicide. It has nothing to do with existence and merit of evaluation system whatsoever and assigning to it responsibility for dying kids is a demagogy at its purest.

>>> And then you assume it is a good proxy for teachers.

Again, I don't assume anything - I say this criteria is inevitable, since the job of the teachers is to educate the children, and well-educated child is considered teacher's success, while poorly educated one is considered a failure. Since we don't have a way to test capability of teaches by supplying them with test kids to be taught in lab conditions - we have to test the actual kids to sample their level of education. It's not a choice, it's a logical consequence of current system of assumptions. It's as much choice as for a man jumping from a building it is a choice to hit the pavement.

>>> Your older education system did a great job.

It did great job for some and terrible job for others. It does the same now, only much more expensively. That is another reason why government is so eager to introduce testing - the costs are raising massively, the results aren't improving, and in some places are actually nothing short of disastrous - with schools releasing people that are completely illiterate. Every school complains it's underfunded but more funds do not seem to produce better outcomes. The budgets have limits, so it is no wonder that whoever distributes those budgets wants to see some metrics about how things are going instead of blindly throwing money in an hoping for the best. If something gets startups killed is exactly throwing money out without any way to measure if something was achieved or not.

Arguing against standardized testing is arguing against A/B testing.

If you can't measure it there is no way to tell whether it's getting better.

I'm sorry, but this is a completely wrong-headed statement. Standardized testing for school children and A/B testing for web site visitors are completely different beyond the fact that both involve the word "test".

When you run A/B tests on a web site you don't tell the visitors which version they are being shown, or even that there are multiple versions at all. That would completely destroy the value of the testing. In psychology research, where ethics generally dictates that the subjects be informed, they are still not told what specifically is being tested, lest they modify their behavior.

On the other hand, everyone involved in school standardized tests knows about the tests, knows about various biases that exist within the tests, and knows what is at stake based on the results of the tests.

The reason that the current form of standardized testing is destructive is pure economics. If I tell you that I will count how many lines of code you have committed to the repository at the end of each week and that your pay will be based on this figure, what are you and your coworkers going to do? You're going to start writing hundreds of lines of comments, putting each variable declaration on its own line, needlessly re-factoring satisfactory code, etc. This is, as I understand it, a well-documented side-effect of measuring programmer productivity using LoC.

Once I implement this evaluation scheme, two things happen. First, you are being evaluated based on an objective metric. Second, you have direct control over the value that metric takes on. Economics, which is, to a great extent, the study of incentives, tells us that you (and your coworkers, at least in general) will game the daylights out of the metric.

So why is it any different when teachers, once told that their salaries and very jobs depend on the results of an annual test, teach their students exactly what is on the test, by hook or by crook, even at the expense of "real" learning?

The problem is that teachers aren't oblivious (neither are programmers). When they see that they are being evaluated based on the outcome of X, they seek to maximize X. So the trick is, if you insist on quantitative metrics (which is not totally unreasonable, at least as the default) then the metrics must be linked to the behaviors that are desirable, not some vague indicator that approximately tracks those behaviors.

Edit: missing word

Not sure if you have ever read about the system in Finland, but its fascinating. And very low on testing from my understanding. Searching for Finland on this education site is interesting http://www.edweek.org//search.html?prx=p&occ=p&qs1=F...

There is a reason why rote learning to some extent works especially in India.

There is so much competition you are likely to inevitably work very hard. So much effort that you will ultimately get it even if didn't want to.

I know this is brute force way of getting things done. But it is still far better than getting nothing done.

> I know this is brute force way of getting things done.

now, theres something to be aware of - "getting _things_ done" in this context means getting a high score in a standarized test, not getting a job.

Having such a score doesn't necessarily mean you are able. Especially in a creative field that requires genuine interest and passion. This sort of training works for training operators of machinery and/or manufacturing workers that require special knowledge, but for creative fields such as programming (and not the CRUD type!) i think having such brute force method of learning is in fact a barrier.

I hear in china that there are so many people who get excellent scores in tests, and yet is unable to find jobs. Simply because jobs suitable for them isn't there, nor are there that many opportunities. The sort of rote learning is really exacerbating the problem, not helping, because the students studied hard (despite not being their "passion"), they get a good mark, but still find themselves in a rathole.

That's mixing up measurement for achievement.

We make mental body builders.

But they make terrible people and thinkers. The idea of educated here is lost. Most of the debate the average person can create is middlebrow at best.

Oh? I can guarantee you that if I went and interviewed any graduates of a high school or university that concentrates on rote learning I could shred any notion that, on average, they had basic competency in the materials they are alleged to know.

The sad truth here is that it is possible to "fake it", at every level. For the most part a person can be productive in even the most technical of jobs without fully comprehending most of what they are working with. People talk about cargo cultism a lot, but in the real world cargo cultish behavior often works. A lot of times just going through the motions is enough to get by.

But that's no excuse to continue to wallow in mediocrity. We can and should do better.

Currently, for reasons that escape me, you lot (Americans) seem to have decided to import those banes of thinking and real learning - standardized tests.

Standardized tests aren't the bane of thinking and learning. Drilling children over and over with questions identical to those on some test is the bane of thinking and learning. The United States has had standardized tests at least since I was a child (I took standardized tests in elementary school almost thirty years ago) and U.S. education has continued to be progressive during that time. College admissions were very heavily influenced by standardized tests when I was in high school, but my classmates and I were not taught by rote, because our teachers did not believe rote learning was effective.

Let me give you a concrete example. My ex-girlfriend grew up in South Korea. She did very well in calculus in high school, where she learned to solve problems by rote. They were shown different kinds of problems to solve, they memorized the steps to solve them by rote, and then they practiced solving those problems. That's all they did: memorize the steps and then apply them over and over to thousands of problems. Then they were tested on exactly the same kind of problems. She got excellent grades and test scores, but she cannot tell you what a derivative or an integral are or why you would use them. I was taught entirely differently. I was taught the concept of the derivative and the integral. I was taught from pictures before I was taught with symbols. The motivation for the concepts was discussed. I don't remember if we discussed it in class, but the textbook even described some of the historical context. We often learned computations in the context of applications: a tank filling with water, a car accelerating, etc.[+]

Do you know what? Calculus was tested the same way in my school system here in the U.S. as in Korea. Each test had a bunch of problems. The more problems I solved, the higher my score. That's it! Nothing creative or touchy-feely or magical about it. Not really any different from the way my ex was tested in Korea. At the end of the course, there was a standard test administered uniformly across the nation (the Advanced Placement exam.)

American teachers routinely invoke the idea of "teaching to the test" to imply that there are bad teaching methods that lead to better test scores than good teaching methods. Basically, they threaten to teach more and more poorly the more their students are tested. The truth is that under pressure to produce good scores, teachers continue to use the techniques they believe are effective. Teachers in the U.S. are taught that rote learning is not effective, so they don't use it. Teachers who believe that rote learning is effective will use it regardless of whether their students are tested or not.

tl;dr My ex-girlfriend and I were TESTED the same way, but we were TAUGHT entirely differently, because our teachers were trained differently.

[+] Of course, it turns out that in order to learn how to do the problems, I had to actually do the problems, and to a limited extent, the more problems I did the easier they became. In the United States education system, we are taught to be horrified by this evil, poisonous art of getting better at something by practicing it, but we still do it. Teachers universally disapprove of it, but they have not figured out how to eradicate it from their classrooms. (Getting rid of testing would be a good first step, I suppose, since then we wouldn't have to worry about whether the students can actually demonstrate competence at anything; we could teach them to attain pure, perfect internal understanding untainted by any outward manifestation.)

You are saying that if you have better teachers who know how to teach, then standardized testing doesn't matter - it doesn't have to be about drills.

That's essentially waving a magic wand. You are super privileging teacher training as some sort of silver bullet solution, but missing the larger system that works actively against the outcome. You are undervaluing the various pressures which get applied to education, one important one being parent desires to see their kid succeed.

It doesn't matter if the kid understands or not. It doesn't matter if he can answer more important questions but isn't able to perfectly prove a theorem.

No amount of teacher training can stand in the face of a single determined parent who wants their kid to ace that test.

Heck a more trivial point which is still important? - teacher empowerment must come hand in hand with training, because each student is vastly different even from him or herself depending on what time of the day/month or year it is. Teachers must be able to make graded and customized judgement constantly. Tests don't allow for that.

Moving the power for measurement into the hands of the test dis-empowers teachers. The test becomes/IS all that matters here, not the education.

And Tests only roughly measure learning. They MOST strongly measure whether you could answer that particular test.

The test is all that matters.

As a result, the kind of teaching you value, becomes deprecated. Its just not economically competitive. As the original article posits. You don't need teachers to pass the test. You need drill Sergeants.

Teachers who are good at that are the ones who will come to the fore, other teachers don't matter.

You could love your subject and instill it in students. But that doesn't guarantee a good grade. Loving a subject, being involved in it, and testing well in it are different. Especially if you don't "think the test".

This is the final pressure - loving a subject or enjoying it is irrelevant. Students care even less, they know that as long as that they get the test - its all good. "I'm getting good marks In a stupid subject like chemistry, who cares?".

Do you want those people becoming your computer science engineers?

The system you are setting up is concerning, and no amount of teacher training will solve it.

-----

Further I think you are talking about standardized tests America had 30 or so years ago. Speaking on behalf of student body of India - we would have loved to have tests like that.

At least in America you had the option of making a life if you had some gumption even if you didn't ace those tests. You were a person as opposed to a Frankenstinian mental athlete.

Our tests were/are compulsory, for many people not in their first language, and they determined everything from your social standing to your marriage prospects. You started studying for your tenth standard exam 2 years before they started, you had no vacation and you had no spare time.

Then 2 years later you would do it for your ISC or IIT or CAT.

All I can say is that we've had standardized tests in the U.S. my entire life, and none of that has happened. I just looked up when the SAT was introduced: 1926. Over eighty years ago! It hasn't ruined American high school education. (If that requires magic, then magic is real.) Of course standardized tests can be misused. If we make standardized tests into tests of rote regurgitated knowledge, if we design tests that reward cramming and mindless drilling instead of learning, if we replace the college admissions process with a single do-or-die test, then the results will be unfavorable. But we haven't done that in the U.S., and we aren't going to.

> Speaking on behalf of student body of India - we would have loved to have tests like that.

Do note, as far as standardized tests go - the SATs were fun - I personally loved taking them and practicing them. They weren't compulsory either. And if you didn't take them, well you weren't going to be a loser in life. There were other things you could do.

Please understand, that without NCLB you could have all those "IFs". With NCLB the dynamic has changed very drastically, and those conditional flags are set.

SATs are very different from NCLB mandated tests to measure student performance mate. They are compulsory and everyone gets graded. (Remember the last time someone ran an experiment where they gave some kids fake good grades and some kids fake bad grades?)

Test structure and test making knowledge isn't static either - it evolves. One of the first thing it starts becoming aware of is people gaming the system - cheating, memmorizing and so on. It tries to adapt to this by making the test more complex or variable. But then to standardize it has to create a (publicly available) rubric for graders to follow. So the process just renews.

You've started down an evolutionary path. Your tests are moving down a particular road. What you see today isn't what you are going to have.

Further I don't need colleges to have a single do-or-die test. - I just need parents to start privileging the test. IT doesn't matter what colleges do unless they work actively against the test. All that matters is that parents start pushing their kids to do well in it. This is one of the engines of the self fulfilling prophecy.

Standardized tests aren't being misused. They are achieving the kind of goals they usually achieve.

Deleted comment

AP Calculus isn't a good example of the "teach to the test" problem. Most elementary/middle and even high school tests aren't nearly as good as AP tests, and most subjects aren't best taught through rote - the best way to learn US history is NOT to be quizzed repeatedly on which year the Missouri Compromise was passed or how many men died at Sharpsburg.

The problem that "teach to the test" introduces is that students focus on factoids and identifying the correct answer in multiple choice, but in the real world (or in college), I'm less likely to have to pick WW2 generals out of a lineup and more like to need to come up with a few paragraphs on what caused it or how it was won or something. Now AP tests do (at least partially) solve for this with their free response and short answer sections, but Standards of Learning (or other state tests) may not.

most subjects aren't best taught through rote - the best way to learn US history is NOT to be quizzed repeatedly on which year the Missouri Compromise was passed or how many men died at Sharpsburg

As far as I know, there aren't any subjects that are best taught through rote. (The confusion between rote learning and practicing skills is a pet peeve of mine.) Learning and testing don't need to resemble each other, not at all, not any more than a physical exam at the doctor's office needs to include a balanced meal, eight hours of sleep, and some light exercise. I think I established that pretty well in my comment: the way I was taught calculus was very different from how I was tested. I was taught calculus through lecture, demonstration, and practice, using applications to motive a new concept, covering the concept intuitively before learning the mechanics, and finally learning and practicing the relevant computational skills. My ex was the one who was taught according to the philosophy that learning should consist of mimicking the test over and over again.

Most elementary/middle and even high school tests aren't nearly as good as AP tests.... Now AP tests do (at least partially) solve for this with their free response and short answer sections, but Standards of Learning (or other state tests) may not.

In other words, standardized tests are capable of reflecting and encouraging progressive teaching ideals OR regressive fact-regurgitation models of learning. I have no doubt at all that poor tests can be written, have been used in the past, and are currently in use in some places. Tests can't help but reflect the beliefs and competence of the people who write them, but that applies to everything else in education. Nothing is immune to being screwed up by individual or institutional incompetence. If you assume competent, progressive teachers and incompetent, benighted test-writers, it's easy to conclude that tests will drag down educational standards. That isn't fair, though. There's no reason to assume that the test designers are any less enlightened and competent than the teachers manning the classrooms.

in the real world (or in college), I'm less likely to have to pick WW2 generals out of a lineup and more like to need to come up with a few paragraphs on what caused it or how it was won or something

I have to disagree with you there. That NEVER happens in the real world. In real life, you most commonly draw on historical knowledge to generate context for things you read or hear. You often have to rely on little factoids such as names and dates to generate connections between what you're reading and what you already know, because you might be hearing something from somebody who doesn't know the context themselves. (Or you might be reading something by a political pundit who is being selective with the facts.) Knowing dates is highly underrated. In a classroom, you learn that dates are the most sterile and inert form of historical knowledge. That may be true for academics or for students, but for laypeople applying a casual knowledge of history in the real world, dates come in really handy. They're the Kevin Bacon of historical connections.

> In other words, standardized tests are capable of reflecting and encouraging progressive teaching ideals OR regressive fact-regurgitation models of learning.

You are putting the cart before the horse. Standardized tests in America circa 2004 allowed for that flexibility. You had an education system that allowed for progressive thought precisely because you weren't forced to standardize.

as I have argued before. Your standardized test earlier, and your teaching system, was NOT comparable in ANY way to what Korea/Japan/India have.

You are now recreating those systems with Standardized COMPULSORY tests. Your comparisons are superficial and invalid.

> . If you assume competent, progressive teachers and incompetent, benighted test-writers, it's easy to conclude that tests will drag down educational standards. That isn't fair, though. There's no reason to assume that the test designers are any less enlightened and competent than the teachers manning the classrooms.

You are painting a far too rosy picture of test taking limits. And part of that reason is because you aren't straying into the details.

Tests can be only varied on a few axes - if you are testing about the revolt of 1857 or the various ways to calculate capacitance there is only a small subset of problems that you can come up with.

Don't forget - These are STANDARDIZED tests - they have to be graded either through an answer key, or through grader judgement - if you allow too much on grader judgement then results aren't standardized. SO it ALWAYS skews to answer keys.

This results in all tests becoming predictable. You do not have that many degrees of freedom when standardizing.

> I have to disagree with you there. That NEVER happens in the real world.

Studying/testing is never about real life. IT can't be. No one knows what sort of life a student is going to lead, so you can only hope to give everyone a similar base of information which should help them find their own paths eventually.

As a result, data has to be produced in long form story format, and tested to see that key pieces of information were retained.

Real life doesn't come into it.