Detecting bots with Javascript for better A/B testing results
kalzumeus.com
kalzumeus.com
What are the advantages of using Rails vs Javascript to do a/b testing? And of course, advantages of either of these "custom" approaches over Google Website Optimizer?
On the general topic of doing A/B testing server side (with Rails or what have you) versus doing it client side in Javascript:
1) Doing it in Javascript sacrifices one-line A/B tests, and that is a cost I'm totally unwilling to put up with because it means I will be too lazy to actually do A/B tests. (Take a look at the framework posted yesterday and what they have to do two swap two divs. That is fairly representative. Note that it gets harder, fast, if you start doing anything the least bit complicated.)
2) Javascript is not your friend for load times. A/Bingo and similarly optimized server-side frameworks are, essentially, free in terms of user-perceptible performance.
That's not true, Youtube used GWO to run a successful, massive multivariate experiment on its US front page, which qualifies as a page of importance in my book: http://youtube-global.blogspot.com/2009/08/look-inside-1024-...
"You might have noticed that I assume humans can run Javascript. (My site breaks early and often without it.)"
I find my sites doing the same thing more and more these days. Do you find that this has caused you any problems? I'm guessing that a large part of your target audience for BCC is non-technical so they're probably a good test...
So, besides companies which restrict javascript (very rare?) there's only those using noscript (very rare?).
More broadly, I tend to think that anyone who goes into their browser settings marked Here Be Dragons is a lot like someone who routes my domain to localhost in their hosts file. Ah, yep, that will indeed cause the site to not work. You want the site to work? OK then, I would suggest not doing that.
Detecting bots with JS knowing they don't run JS would be quite challenging.
For an example, http://code.google.com/p/wkhtmltopdf/ is an application which converts a webpage into a pdf and supports javascript.
Nice post Patrick!