Back to the topic, we don't make much innovation on the model, so I am probably not the best person to evaluate how a model compares with SOTAs. There are indeed lots of super cool techniques being explored lately that makes it possible to deploy smaller and smaller models, for example, LLM.int8() [1] and int4 quantization [2] without loss of zero-shot accuracy. Can't predict the future, but maybe one day there will be something really powerful but small enough to fit in the pocket of everyone :-)
[1] Dettmers, Tim, et al. "LLM. int8 (): 8-bit matrix multiplication for transformers at scale." arXiv preprint arXiv:2208.07339 (2022).
[2] Dettmers, Tim, and Luke Zettlemoyer. "The case for 4-bit precision: k-bit Inference Scaling Laws." arXiv preprint arXiv:2212.09720 (2022).