They report 73% accuracy on CIFAR-10 using Alexnet model. That's very low (should be at least 85% even without any augmentation or regularization). That result puts the rest of the paper in a bad light.
Okay, although the paper is more useful as an educational guide which is also why it was posted. Afaik, Bayesian techniques have never been victorious except potentially for hyperparameter tuning.
I agree this paper is more like a tutorial, which I like, but that 73% accuracy is from their implementation of a standard (non-Bayesian) Alexnet, so no excuse there.