Hacker News Reader

Comment by salimmadjd | Hacker News Reader

This is cool. However a few points:

1 - Most small variations on A/B landing pages do not show conclusive uptick. You usually need to do drastic changes to see a big difference. I wonder if the managing one code it would be messy or is there a way to have two or more different files?

2 - Many organizations have a problem understanding how much better one form is performing if they have low volumes. Is there statistical presentations that with C confidence can tell A is better than B.

3 - Sometimes is great to do A/B/A or A/B/C does your tool provide that?

View on HN

1) One of the advantages of Airfoil is that you can significantly change the design and structure of your page, rather than just superficial changes to copy, colors etc. In my opinion it's much easier to make drastic changes directly to the HTML rather then messing around with an in-browser WYSIWYG HTML editor. Of course you can also use our variant tags to change what javascript is loaded, drastically effecting the content of single page apps.

2) We rolled out two variants for the headline copy on airfoil.io, and even in our relatively small sample set (less than 2k uniques so far) we've seen a dramatic difference in conversion (25% vs 10%)

3) You can add additional test cases, and change probabilities, by editing the app's config.yml file. The relevant configuration might look like this:

  variants:
    a: 0.2
    b: 0.2
    c: 0.6

>3) You can add additional test cases, and change probabilities, by editing the app's config.yml file.

Nice! Will your dashboard take the ratios to account when helping the user see which design is performing better and also show the confidence interval?

Backlift's analytics dashboard is simple and useful for quick analysis and tuning during a launch. In addition all the code for the dashboard is in the /admin folder, so it can be extended if needed.

So I guess the answer is no?

Reply on news.ycombinator.com