What are the advantages of using Rails vs Javascript to do a/b testing? And of course, advantages of either of these "custom" approaches over Google Website Optimizer?
On the general topic of doing A/B testing server side (with Rails or what have you) versus doing it client side in Javascript:
1) Doing it in Javascript sacrifices one-line A/B tests, and that is a cost I'm totally unwilling to put up with because it means I will be too lazy to actually do A/B tests. (Take a look at the framework posted yesterday and what they have to do two swap two divs. That is fairly representative. Note that it gets harder, fast, if you start doing anything the least bit complicated.)
2) Javascript is not your friend for load times. A/Bingo and similarly optimized server-side frameworks are, essentially, free in terms of user-perceptible performance.
That's not true, Youtube used GWO to run a successful, massive multivariate experiment on its US front page, which qualifies as a page of importance in my book: http://youtube-global.blogspot.com/2009/08/look-inside-1024-...