At the same time, possible since it's only classification tasks. I mean, the method explained is technically plausible, a lot of people thought about it, we were just unable to find a method to do so.
Very unlikely true, unfortunately.
I mean, the process should have been to contact some local academics to discuss the mater. If I say it works (or it doesn't) I'm adding near nothing to the claim, as I'm not an academic myself.
Big claims like this need clear and solid work. Here it just looks like LLM generated.
It's OK to call out fake claims. But it requires going through the process if such is reasonable, it just seems to take a couple of hours to find out.
What they achieved is to create tiny student models. Trained on specific set of input. Off the teacher model's output.
There is clearly novelty in the method and what it achieve. Whether what it achieve would cover many cases that's another question.
If you look at the student-only scripts in the repo, those runs never load the teacher. That's the novel part.