EELBERT: Tiny Models Through Dynamic Embeddings
arxiv.org
arxiv.org
https://arxiv.org/abs/1909.11687
https://arxiv.org/abs/2210.07904
I wrote an article on the subject earlier this year - https://neuml.hashnode.dev/train-a-language-model-from-scrat...