Compressing neural networks for inference is an entire subfield of work, with a number of effective approaches. It looks like the paper doesn't compare against any of them?