Oh yes, absolutely. We only make claims for "natural distributions and models" (^).

It's almost certainly possible to break this by pathological choice of a data distribution or model/initialization/optimization scheme. But, I don't think this is interesting -- what I think is interesting is that this seems to hold true in real life, in "natural settings".

---

(^) Whatever "natural distributions" means... (which I think is a good research direction in itself).