wow this could make the fabled 65 billion parameter llama sparsed and pruned runnable on a 3060
> at minimal loss of accuracy
Suggesting that there is a lot of redundancy in the weights.
As some other commentator stated, there's currently a lot of low hanging fruit in optimizing NN.