It's actually a tad annoying because there's no way to do a "nofollow" in javascript.
It's actually a tad annoying because there's no way to do a "nofollow" in javascript.
On some sites what I do is make a fake virtual directory and just ban the directory in robots.txt while prefixing the links with the fake directory path.
Oh and I swear google sometimes follows stuff in robots.txt, it just doesn't index it. Then there's the "alternate" user agents they use like pretending to be IE (from a real Google IP).
Exactly. Right to your special page that earns a 24 hour ip ban.
It doesn't take that long to code and should keep any bot away.
I'll also note that I've never obfuscated my email address (chris@improbable.org) and spam arrived in roughly equal volumes there and to less public addresses which were guessable (except for test accounts created with MD5 hashes[1]). People do harvest addresses from web pages but there are enough other ways spammers get addresses that you need a robust filter in any case; once you have something like SpamAssassin or Google's filters, the value of keeping your address private disappears.
[1] That also yielded the interesting datapoint that hotmail used to sell addresses to spammers in early last decade as the first spam arrived within 20 minutes to a long, random address created with the "don't publish my info" option.
My approach is a little more convoluted than simply document.write or something like that, so maybe that matters. I get crawled by bad bots all the time, but these addresses never seem to get spammed. The same is not true for addresses displayed in plain-text somewhere.
My explanation has always been that there are so many plain-text addresses on the web that it isn't necessary to add JavaScript parsing logic to most harvesters. Much like car alarms or door locks--you don't need to make them unbreakable, simple harder to get at than the one next door. Someday we may need to start displaying email addresses as (or behind) captchas.