If I think about it, I find it awful. The fact that we need to put junk in our own stuff just for crawlers does not sit well with me.
Any functioning society would deal with the offenders directly and had this stopped before it became an issue for most sites.