Our hypothesis was that students self studying for the SAT used books (eg Kaplan) but the solutions in the back were poor in explaining how they arrived to the solution because they usually just cut to the chase rather than explaining the approach & helping you arrive to the solution.
So we took each practice SAT math question and wrote out a solution spread over four swipeable pages. Our hope was that if you saw the first or second page, you'd arrive at the solution without having to see the answer, thus increasing your chance to solve it on your own next time.
We spent over 100 hours tutoring kids and had them use our MVP to mild success. The obvious next step was to automate the solutions rather than hand write it ourselves - like GPT-4 has done here.
As someone who has logged over 2000 hours teaching, there's a lot of soft nuances and tactics to tutoring. Also, motivation is a big thing a good tutor brings. I don't see this solution as quite there yet but getting close.
Very optimistic here.
So why are there still so many bad study books?
This may be the most transformative change to education since the invention of public schooling.
But EdTech hasn’t really delivered much value. The modern LMS feels to me like a reinvention of Facebook groups in many ways…
LLMs feel like something that _could_ very well end up eating education, and that’s a really exciting prospect. My personal experience with schooling was less than stellar, and figuring out alternative and more scalable ways to get knowledge into kids’ heads has a lot of possible upside for society imo
I agree with your sentiment that there's a lot more potential value to be derived from EdTech than we've seen so far, but even if LMS's are "just" like Facebook groups, that's already the equivalent of hundreds of billions of dollars in economic value.
When I was a kid, if I was sick for a few days, I had to manually copy my friend's notes after I returned to school, and would still feel confused. Now, with an LMS, I have access to all the learning content from home, for my own class, and for resources from similar classes taught by amazing teachers from across the world, at any modality I prefer. This on its own is incredible value. (although with the caveat that being sick is no longer the excuse to rest and relax as it used to be)
I think general public having access to these AI tools is going to create a shift from scholastic aptitude testing to more performance testing. So no more homework, you use your time out of class to train so you can use the in-class time to perform based on the training you've been doing with your personal AI tutor.
Assuming the divide will be real, you can just record a million problem solutions from ExpensiveLM and use it to tune your CheapLM, it will be a specialised cheap model that works ok on your task of interest.
I think you'd find that even by the sat-prep age the actual limiting factor for many (most?) of the tutor-less is a fate-accompli attitude towards their path in life.
Might as well turn on the chat bots at the cradle. Ah, and make sure everybody gets the same quality AI to parent them for it to be egalitarian.
I do think there are valid concerns that this technology (like old-school human tutoring) could be disproportionately available to already-privileged children. This could potentially widen the achievement gap. But it seems way more likely that we can get poor kids a laptop to tutor them vs. getting them a dedicated human tutor.
Everyone has chatgpt here but not everyone will use it. Just like how not everyone shows up to office hours or asks questions of their teacher.