jillbore··on Fast Llama 2 on CPUs with Sparse Fine-Tuning and DeepSparseThis is about seeing inference benefits from unstructured sparsity.
jillbore··on Sparse LLM Inference on CPU: 75% fewer parametersThat is the accuracy of the non-finetuned model i.e. mpt-7b. Once finetuned, it can get around 30% accuracy on 0-shot. Not sure what 8-shot performance is for the finetuned model. https://arxiv.org/pdf/2310.06927.pdf