Yeah good perspective -- I guess I was thinking about this more from the perspective of predictive modelling than science.

Model averaging can be quite useful when you're averaging over versions of the same model with different hyperparameters, e.g. the number of clusters in a mixture model.

You still need a good hyper-prior over the hyperparameters to avoid overfitting in these cases though, as an example IIRC dirichlet process mixture models can often overfit the number of clusters.

Agreed that model averaging could be harder to justify as a scientist comparing models which are qualitatively quite different.