I use a sub-domain (e.g. @m.mydomain.com) for my catch-all and this hasn't happened to me. There are various easy ways for spammers to find out about domains, but sub-domains can remain relatively obscure.

Or worse, using <randomchars>@somesubdomain.example.com as the SENDER address on spam to others. I once had spammers find a subdomain that accepted wildcard emails, and the backscatter was just insane. Had to spend a whole day trying to make a list of valid <usernames>@ on that subdomain to whitelist to put an end to it. (Not easy if you haven't already been keeping track of which addresses you've handed out throughout the years)

I have a setup like that, I get more spam because of that of course but Google is really good at filtering it. And especially in conjunction with priority inbox, it's a breeze (and very convenient).

Yup. I used to do this with a personal domain. After a while I realized that the spam folder was filling up faster than I could manually empty it!