Hacker News Reader

So much this, even if you can create a decently accurate ML system to score student essays it absolutely doesn't handle the adversarial nature of scoring and is frankly insulting given how much students are paying for tuition these days. The problem is AI-complete.

Its a good tool for feedback and its acceptable for a MOOC or low-cost courses as long as there are adequate disclaimers and the student's final score is determined by a properly human graded assignment or exam.

I used to tutor kids in AP US History and World History when I was a senior and freshman/sophmore in college. The key success factors were getting good scores on two essays and a document-based longer essay where you would use primary source material provided in the test booklet.

These tests were all about test strategy. The written material was graded (in the 90s) by tables of people basically working a checklist for a few different things. If you had insight into how they graded (our teacher had been a grader and had our class start grading each others work when I took the class), you could get a great score even in an area where your knowledge was weak on the essay questions. Another trick was to know 3-4 general concepts from a well regarded historian to support whatever conclusion you cook up.

I think you could score these with NLP with a reduced number of humans to fine tune it. (In the 90s, I think up to 4-5 people would touch a paper) The exam would get dumbed down over time as a result as the test developers would narrow the scope of the exam to make scoring better, but I'm sure that's happening anyway as the testing vendors shave cost.

> Working in NLP, it really bugs me when people overstate what can be done like this.

But they have to. Otherwise funding goes away.

> It creates massive expectations that are bound to fail, and then the entire field suffers.

Yes. But by then the people overstating now will have made their money and moved on.

I agree with you, but honestly companies and investors are so shallow it isn't hard to see what is going on.

That is fraud.

   > Yes. But by then the people overstating now will have made their money and moved on.

Yep, and those people will likely be sending THEIR kids to elite private schools with 5:1 student-teacher ratios and 20-40K/year tuition for elementary and high school. These schools will DEFINITELY NOT use "AI" for grading.

It's weird that what is luxury today was once the norm: organic produce, a stay at home parent, natural (as opposed to synthetic) clothing, etc.

I won't be surprised if having human teachers / caregivers will be reserved for the upper classes.

If you think you can get a 5:1 student ratio on 40k a year I have a bridge you can buy. For a 10:1 you'd be looking at 100k.

>Others have pulled one over on the computer by padding their essays with long quotes from the text they’re supposed to analyze, or from the question they’re supposed to answer.

This is a long standing tradition for students on pretty much every humanities essay.