Hacker News Reader

"I have probably told a hundred anecdotes like “I just did an A/B test and increased software sales by 70% with 99% statistical confidence. The change was a two-character configuration tweak that I dismissed on a hunch six years ago.” (That totally happened this May. Ask me for details later.)"

Would now be a good time to ask for details?

View on HN

It's worth talking about in a wee bit more detail, but

@new_user.card_limit_for_free_trial = ab_test("user-card-limit-may-2012", [8, 15], :conversion => "purchase")

gives away a bit of it. (That's literally all the code. The old default was 15.)

Funnily enough, I mentioned I might do that test eventually on HN ( http://news.ycombinator.com/item?id=3864079) and at the time panned its likelihood of success. Shows what I know, for the 495th time with A/B testing.

Interesting. I remember you talking about how it likely wouldn't make much of a difference, because almost all parents have less than eight kids, and almost all teachers have more than 20, so the number of people who'd be induced to buy the software by an eight card limit shouldn't really be much higher than the number who'd need more than 15 cards, anyway. Any theories on why it ended up mattering after all?

Reply on news.ycombinator.com