Okay, this is at least partly disingenuous,pruning deep nets absolutely helps (see e.g. Song Han's Deep Compression paper). It's also very worrying they only test on the toy dataset, MNIST.
Thanks for that, by the way. I have a friend who can really use that information.