In the book does he reflect on any of this, or is it based on the MIT challenge at all?
Anecdotal, but I burned through 1/3 of a two semester abstract algebra course in 3ish days of full-time studying, and solving all exercises. But in all honesty, the retention would have been very low had I not began a linear algebra course aimed at graduate pure math students (I am not a math student, nor do I have a math degree).
For such a challenge to work with topics like mathematics, the content needs to be planned such that every course studied builds on top of the previous one, so that the student essentially revises and uses the content studied the previous week.
Have you tried flashcards and spaced repetition? Perhaps these could fix what you perceived as a downside?
By the time of the examination, I have an extensive set of notes that I can search through, and have transitioned to solving the exercises for speed over precision since precision has already been attained. I also transition from solving on paper to solving in my head.
By solving for speed, I mean that after many repetitions the answer that I provide is coarser and distilled because I have good understanding of the finer details, and the finer details can only be attained by writing extensive notes and solving for precision.
In the end of the day though, time is needed to fully absorb the content. The reason is that making structural changes in the brain is very expensive, but spaced repetition and usage of certain pathways make them much more efficient.
In essence, the Feynman Technique is, inho, the best way for a scientist to self-study a topic, and flashcards in various forms along with spaced repetition help achieve that task.