Hacker News Reader

As an alternative, I wrote a port of Vanity for PHP that we use for A/B testing. It uses redis to store data

http://github.com/aliasaria/PHP-redis-A-B-Testing

This is an honest question: is REDIS the best option for storing A/B tests or is just a matter of taste? MongoDB or even MySql could work?

I try to keep most of the A/Bingo work in a cache store (Redis, by choice) because sites with traffic two orders of magnitude or so higher than mine would blow up if they used MySQL for it. At something like Facebook game scales I think it would be smart to take A/Bingo and rip the database out entirely.

Writes hurt. A/B testing requires, minimally, O(tests * participants) writes. (And more reads, but the disk will be on fire from writes first.)

However, despite the cost, A/B testing benefits minimally from ACID. If you lose 2% of writes, oh well, it comes out in the wash anyhow. Thus the huge hardware/maintenance budget you'd need for web scale A/B testing on an ACID architecture makes little sense.

Reply on news.ycombinator.com