One problem with AI in education is that it promotes magical thinking. That is, if you rationally understand what the AI is doing, it seems awfully simple. And then the disappointment comes: if it is simple and understandable, it isn't really AI.

That's why you see a race for obscurity -- lots of jargon to describe the complexity of the system.

In reality, good educational software is about good design, good goal alignment, good assessments, good sequencing, good intervention material -- it's all based on classic instructional design, not a superior algorithm. Has the software been set up to support data driven continuous improvement? That is more important than an algorithm.