Is there some principled reason to believe that an AI trained on 8th grades will emergently perform better than 8th graders?
The point is that the AI is already fully trained on teaching. But it uses the student's learning progress to determine how that specific student learns the most efficiently.
Yes, possible dystopian outcomes abound, maybe they can be mitigated.