Strong disagree.
In a completely unrelated field, I'm seeing pretty great results with knowledge distillation to generate AI language models that fit into 20 mio float16 parameters (40MB) and can process 1GB of raw text in roughly 12 hours.
In a completely unrelated field, I'm seeing pretty great results with knowledge distillation to generate AI language models that fit into 20 mio float16 parameters (40MB) and can process 1GB of raw text in roughly 12 hours.