I am working with reducing the time teachers spend on exams and assessments. I have access to a cleaned and manually scored dataset of 550k essays that is exponentially growing. Looked at creating at a model based on this dataset to automatically score essays with NLP parameters such as grammar, structure, spelling, word complexity, sentiment, relative text length etc.

The problem that I encountered was actually how to apply it in a useful way, since the problems mentioned in the article are quite obvious when you design the model.

Options that I saw:

1. Use it as autonomous grading with optional review by the teacher, see the linked article for the problems with this.

2. Use it as a sanity check on the teachers manual scoring, but it would not reduce the work load and probably just undermine the teacher.

Do you have any suggestions for how such a model could be applied in a practical and ethical way?

Had some thoughts on how to measure actual knowledge about a subject, but that would require a massive knowledge graph which would introduce a huge amount of complexity just to see if it would be a feasible approach.