Comment by raganwald | Hacker News Reader

Comment by raganwald | Hacker News Reader

Parent Full thread

The writer's complaint is daft. The purpose is to test if you can read a passage and comprehend it. Has nothing to do with microscopes.

From my understanding of cognitive science, the question asked is deeply flawed if you want to test comprehension without testing knowledge of microscopes.

One approach to answering questions like this is to build a little model in your head of what is being described. You then answer questions about the model in your head. Experience with microscopes or similar devices is incredibly valuable to building such a model in your head and answering questions about it. This was one of the “big problems” in first generation AI: to answer questions, a computer must be able to construct models, and to construct models, a computer needs a huge amount of real-world experience.

What I’m saying is that answering this question well requires experience with microscopes or similar devices. Being able to answer the question without prior knowledge of what a microscope is, or how light behaves, or lenses period, or refracting telescopes, and so forth, is ridiculously hard.

Now, there may be a good reason why an entire test full of a variety of such questions is effective for accomplishing whatever the school board wants accomplished—getting funding, I imagine—but I stop well short of saying the author’s complaint is “daft.” I think there are good reasons to be curious about the biases inherent in this question, and about where schools end up if they build their entire approach to teaching around scoring well on questions like this.

> One approach to answering questions like this is to build a little model

This test question isn't testing that.

> What I’m saying is that answering this question well requires experience with microscopes or similar devices.

It sure doesn't. It requires you to be able to read and comprehend text. That's it. The subject could have been Martian fizzbozzles instead of microscopes.

> Being able to answer the question without prior knowledge of what a microscope is, or how light behaves, or lenses period, or refracting telescopes, and so forth, is ridiculously hard.

It sure isn't. Anyone who can read and comprehend text can answer it with no prior knowledge of the subject.

My HN friend, my argument was that the writer isn’t daft, and neither is his complaint, and I stand by that. My argument is that the writer raises interesting and useful questions, and I stand by that too.

As to the fact that the question doesn’t test internal modelling, of course it does. If you made it a question about something nobody has ever seen before, then it wouldn’t test knowledge of microscopes. But it does ask about microscopes.

Perhaps, in aggregate, the test covers enough subject areas that the students tested will be exposed to a roughly equal mix of things they have experience with and things they don’t. But as I noted elsewhere, the fact that it’s possible to answer this question from the text alone in no way changes the fact that it’s easier to answer the question with prior experience. And from that, I think it’s perfectly reasonable to ask whether this is a good question.

Yes, if it were truly just about reading comprehension, then the test-writers would be horrifically incompetent, and therefore unfit to test anyone. Anyone who competently designs tests (experimenters, pollsters, etc) knows to address such biasing factors.

But I think the testers aren't total morons, and know that such a question is biased towards children with proper (privileged) backgrounds where microscopes and the respective training are commonplace. They'll be faster at answering the question and not feel ignorant — crucial advantages on tests.

(The testers may not even like this facet of their jobs, like ad industry developers who use ad-blockers, but it sure beats unemployment.)

And the whole question smacks of "scientism", where tools are confused with science. You never see a question like, "Scientists have believed something for 200 years. Is it necessarily true?"

> It requires you to be able to read and comprehend text. That's it. The subject could have been Martian fizzbozzles instead of microscopes.

And therein lies a great fallacy which is still widely believed in our post-Enlightenment world. But the map is not the terrain. A mental model cannot be completely represented with words or pictures, or anything. Words and pictures, even good ones chosen for pedagogical value, are only pale projections of a complex thing onto a flat surface.

>Anyone who can read and comprehend text can answer it with no prior knowledge of the subject.

Without qualification, this statement is strictly false. There may be certain kinds of text that can be comprehended with no prior knowledge of the subject if you assume a certain baseline of internalized knowledge. There are many old parlor games (which have sadly gone out of style) which rely on hacking this qualification. For example, one of them requires that one person follow the instructions of another. One person tries to describe how to draw a particular geometric figure. The person doing the drawing must follow the instruction, but they also are free (in fact encouraged) to ignore the intention of the instruction. It's a game that puts the difference between words and reality up-front and center.

You put this wonderfully. People are very little more than our experience with the world itself. Without external stimuli, we would be trapped within our own minds and only be capable of evaluating a limited set of possibilities. Merely the act of touching an object generates complex cognitive experiences. This is not possible by abstraction alone.

I understand what you're getting at, but that's not the case in this question, since one of the four answers are stated almost word for word in the text. (caution point 4)

The rest of the alternatives are "plausible guesses" which you might think are reasonable if you don't remember the point above. So what they're REALLY testing is if you read the text carefully enough to remember that line or not.

Actually having knowledge of microscopes are probably a bad thing on this particular question since you're likely to skip over the boring warning part, like I did when I quickly read the question just now.

> Anyone who can read and comprehend text can answer it with no prior knowledge of the subject.

That's the writer's complaint. In order for this question to be testing reading comprehension rather than laboratory experience, it must be assumed that the test taker has not used a microscope. Otherwise, they could answer the question without even looking at the text.

Did you read what raganwald wrote? He's saying that reading comprehension is not typically about memorization, it's often about model building. As a human reads they don't just fill up big bin with data, they build up models. When reading a narrative we will build up ideas of characters. We'll fill in more and more of their details. And when probed about the story we will fall back to our knowledge of those characters, and it may take us a while, if we even can, to figure out exactly which events in the narrative contributed to each aspect of a character.

In this case the object of the narrative is a microscope. And as people read through the text they aren't memorizing sentences or facts they are filling out the model, or character, of the microscope. But people who already have an accurate model of a microscope from personal experience will have an easier job doing so. It's similar to the way standardized tests can be racially biased. If I put forth a "story problem" about the rules of football, being familiar with those rules already gives one an advantage.

> It sure isn't. Anyone who can read and comprehend text can answer it with no prior knowledge of the subject.

This reminds me of people who think the manpages are a sufficient resource to completely understand the unix cli environment.

It is a reading comprehension question. Read the text provided: "To avoid crushing the glass slide when focusing, begin with the lens close to the specimen"

The microscope lens should initially be placed close to the glass slide: C. To avoid breaking the glass slide when adjusting the focus.

"The microscope lens should initially be placed close to the glass slide: C. To avoid breaking the glass slide when adjusting the focus."

Incidentally that statement only sense if you know which direction to turn the focus knob, and you recognize the fact that the hypothetical microscope is capable of crushing the specimen. I had one when I was a kid that was incapable of doing so.

I answered the question correctly, but it only due to my cynicism, not my reading comprehension.

Blah blah blah blah. "To avoid crushing the glass slide when focusing, begin with the microscope close to the specimen and then gradually back off to focus." Blah blah blah blah.

Utterly incomprehensible!

And you know what? I'll tell you why people get this wrong. It's because their brains turn off. They say "omg microscopes scary! big block of text!" and guess at answers instead of actually bothering to try and read the text. That's one reason why they get poor scores at reading comprehension: a lack of reading with intent to comprehend. Heck, if they're pressed for time and the guessing-wrong penalty is low enough, it's an entirely rational response.

The fact that the journalist is that dense is mostly a symptom of "oh hey I've found an argument against standardized testing to fit my preexisting conceptions of the matter! Screw reading it, it looks complicated, I'll just use it to justify things." Same basic problem.

>It's because their brains turn off. They say "omg microscopes scary! big block of text!"

s/microscope/computer and you have the cause of an average Helpdesk's stress...

ed. Downvotes? Really? Nobody has ever struggled with a problem which could be solved if the help-ee has bothered to read the text on their screen?

Yes. My mom said "Now what do I do?" when there was a box on her screen that said "Click Next to Continue" and that's all that could be done. I've been trying to teach her to read the boxes that come up before she calls me for help.

On the other hand, when she doesn't call me for help, she ends up printing off a picture from a webpage and scanning it back in to get it into a power point presentation. That's better than giving up on the problem, I guess.

This is why I think the time limit on such tests should be eliminated. For many well-paying occupations, correctness is far more important than speed.

I saw that too. It doesn’t change the fact that understanding of the domain strongly biases the result of answering the question. What you’re saying is that it is possible to answer the question while staring bewildered at a device you have never seen or interacted with. What I’m saying is that it is much easier to answer the question if you have hands-on experience.

And I think what the author is saying is that we should spend more time giving kids microscopes and testing what they see with them.

TANGENT ABOUT BIKE SHED COLOURS:

Questions like this remind me of questions about OOP. First the preamble: “Organize objects into classes to create a hierarchy of abstract types....” Then a question: “Classes are used to (a) provide a constructor for objects, (b) to promote reuse of behaviour, (c) to model a type hierarchy, (d) to facilitate modular software development.”

You ask someone to build a model in their head, and then the most successful strategy for answering the question is to turn the model off and use brute force to scan the exact words of the question, because the more knowledge you have of the domain, the less likely you are to answer the question correctly from your experience.

What you’re saying is that it is possible to answer the question while staring bewildered at a device you have never seen or interacted with.

Based on my life experience so far; this is one of the most important skills to learn.

What I’m saying is that it is much easier to answer the question if you have hands-on experience.

Of course; but that isn't the point of the test.

And I think what the author is saying is that we should spend more time giving kids microscopes and testing what they see with them.

Yes, but again this test is not about that. And if the author is arguing that practical experience should entirely supplant this sort of critical thinking then I entirely disagree.

Critical thinking is taught from a young age, in very subtle ways. The ability to approach a topic you know nothing about, dissemble it, and answer a question (either posed to you, or by you) is fundamental to all of our lives.

off topic: i hope you meant 'disassemble' :).

'dissemble' means "to put on a false appearance : conceal facts, intentions, or feelings under some pretense" according to http://www.merriam-webster.com/dictionary/dissemble.

I agree with everything you are preaching about critical thinking. Personally, I don’t think this particular type of question is good at testing critical thinking, and worse I think that the incentives in the school system are all about gaming the test rather than teaching critical thinking.

My feeling looking at this is that if I wanted kids to get the highest possible score, I’d teach them how to take tests rather than buy them a bunch of tools and have them figure out how to use the tools from the instruction manuals (reading, critical thinking, problem solving). Whereas if I wanted them to learn critical thinking, we’d be solving problems with tools.

But I accept that this is a very large topic, it’s not like this is a one-question test, and students are exposed to a variety of stimuli. Also, there is always the democracy argument: This might be the worst test ever devised, except for all of the others we’ve tried. Perhaps this is the “least worst” way to test and teach given other factors in play.

> What you’re saying is that it is possible to answer the question while staring bewildered at a device you have never seen or interacted with.

The question is completely invalidated by knowledge of the domain. If anything, they should have constructed an artificial device that would guarantee the student didn't have domain knowledge that would allow them to bypass the necessary critical thinking:

To avoid smaggling the fizdoodle, make sure you begin with the smunderson in the upright position.

Q: Why should the smunderson start in the upright position?

A: To avoid smaggling the fizdoodle.

edit: s/being/begin/

So wait, you're saying the reason for the lens to begin close to the glass is the same reason the lens should begin close to the specimen? How did you come to that conclusion?

Context clues from the "How to" section steps 2-6.

Or you already knew how a microscope works, and performed that step implicitly like the rest of us did.

When I was taking these sort of tests, I learned to delay processing of non-question text until I've read the question. Reading comprehension, in most of the standardized tests I've taken, was nothing more than ability to quickly pattern match keywords from a question to unstructured data preceding it. This kept me from building models of questions in my head and thusly prevented any preconceived notions or facts divert me from the answer the test considered correct.

Reply on news.ycombinator.com