Hacker News Reader

Comment by danielford | Hacker News Reader

I teach community college biology, and I agree that we're really bad at teaching critical thinking. But the Collegiate Learning Assessment (CLA) cited by this article was graded by a computer last time I checked. Here's a direct quote from one of their papers a few years ago:

"Beginning in fall 2010, we moved to automated scoring exclusively, using Pearson’s Intelligent Essay Assess or (IEA). IEA is the automated scoring engine developed by Pearson Knowledge Technologies to evaluate the meaning of text, not just writing mechanics. Pearson has trained IEA for the CLA using real CLA responses and scores to ensure its consistency with scores generated by human raters."

Link below: https://www.pdf-archive.com/2017/06/06/cla/cla.pdf

Most of you are more knowledgeable about technology than I am. So I'll leave it to you to decide if using an algorithm to grade an essay-based exam of critical thinking is a valid approach to this problem.

View on HN

Leave it to Pearson to sell us the problem and then sell us the solution. Taking poetic license to exaggerate just a bit...

The problem: High school kids now spend 100% of their time studying prepared curricula, sold by Pearson, to study for standardized tests, sold by Pearson.

The result: Students lose critical thinking skills.

The solution: A standardized test for critical thinking skills, sold by Pearson, and of course a prepared curriculum to study for the test.

As a computational linguistics grad student I find Pearson's "product" line completely mind-boggling, and their peddling it deserving a giant class-action lawsuit. Consider that the state of the art in machine representation of words is something around Word2Vec or GLoVe, and that we have some okay dependency parsers. That their system provides scores consistent with human raters is likely just evidence that they have a coarse-grained and noisy human evaluation system.

So as long as you think critically the same way as everyone else does you'll be fine.

Or perhaps worse: think critically the same way the test writer does: http://www.huffingtonpost.com/entry/standardized-tests-are-s...

I've often thought that a lot of the high-brow analysis put into art was junk. Just people taking dots and connecting them with shreds of evidence existent in the art. Confirmation bias masquerading as analysis. It's nice to see an artist agreeing with that viewpoint.

I should clarify that I don't mind when the context of a piece is explained. I like knowing about where the artist was when a work was created; what was happening around them that might have influenced the work. It's when that jumps to "and this small detail is about this particular thing that was happening" -- and always spoken with confidence -- that I feel like the train jumps the rails.

>> It's nice to see an artist agreeing with that viewpoint.

Yep. I've always beaten myself up over my ACT score. Near-perfect scores on grammar, science, and math, but near-zero on reading comprehension. And it was a lot of, "what is the author trying to express by using this word in the title?" I'd rather know how good I am at, "after reading this 5 page article, did you catch this really important detail well enough to recall it quickly?"

Art's important too, but can you judge someone's artistic side in a multiple choice test graded on a scale of 1-9? Don't think so...

Ha! I don't know if you've ever seen Ocean's 13, but there's a line in that movie that cracks me up along the same "high brow" analysis lines.

> Matt Damon - "Do you have any wine back there?

> Lady - "Château d'Yquem?"

> Matt Damon - "As long as it's not '73..."

Just makes me chuckle every time because he's a con artist in such a broad field almost nobody can actually identify all of the good and bad varieties from any given year. By just giving an obscure reference you somehow sound like you know what you're talking about...knowing that nobody else actually knows enough about what they are talking about to call you on it.

Just struck me as a great bit of "high brow crowd" humor.

Haven't seen the movie, so it's hard to directly comment, but for what it's worth, Château d'Yquem is a very famous wine. Exactly the (rare) sort where the popular wine magazines will routinely every few years have an article reviewing how the different historic vintages are holding up -- should you drink that 1975 d'Yquem now or hold it a few more years?

It also would be a very dangerous wine to BS about if you didn't know anything about it. 1973 apparently was a lesser year. (Still, it would run you something like $500 a bottle today.) 1975 and 1976 were classic years, name those as something to skip and people will be questioning your taste. And they didn't release a wine at all for the 1972 and 1974 vintages, because they didn't think the grapes were up to snuff.

I had to look those details up because I haven't paid much attention to wine in 20 years. (Wife doesn't like it, so it's hard to justify buying even $20 bottle. Not that I ever could have afforded a Château d'Yquem.) But I still remembered the mid-70s produced a couple of really good vintages. Someone who was actively into wine could probably have given you all those details without any research.

The fact that it claims to check spelling and grammar seems suspect to me because if it really were even at least as good as Microsoft word at good at checking spelling and grammar they would have spun it out and sold it as a spelling and grammar checker instead of as a complete packaged writing analysis tool. This makes me doubt the validity of their more ambitious claims like checking for quality of "ideas" and analytics.

It seems to me that there is a much easier way of automating logical reasoning tests. Just make it a standardize multiple choice and have a machine check the answers - the LSAT is probably one of the most successful analytic and logical reasoning test and it has been done this way for a while.

Reply on news.ycombinator.com