The critique is about the importance of priors in BNN. In my humble understanding of Bayesian reasoning the argument to defend any prior is that with enought data the learning method converges to the real distribution, so if the result of any learning method depends heavily of any prior assumption then that assumption is crucial and in no way can it be taken randomly. On the other hand, it is well known that deep learning can learn any random model, so in the end I think all of this is about the bias-variance trade off. If your prior has an infinite number of adjustable parameters (zero bias) then the variance becomes infinite ( your result will depend and becomes equal to the training set).

So in practice one should choose the prior with the minimum number of parameters (bias) that shows a good learning performance on the available training set.

Anyway, trying to measure how a prior generalizes or not in BNN seems to be another way of thinking about bias-variance, if there is more than this, I would like to know.