HNHacker News
TopNewBestAskShowJobs

kylerush

610 karma · joined May 4, 2011

http://twitter.com/kylerush
submissionscomments
kylerush··on Why and how to avoid hamburger menus
The hamburger menu experiments by James Foster are interesting, but the problem is that they assume the primary goal is navigation clicks. Rarely is a navigation click a bottom line business metric for a company. Almost always you want the user to convert into a customer for whatever you are selling. A more compelling experiment would be one that shows results for both a primary action of new visitor to customer as well as nav click conversions. The fact that fewer users click on the navigation is not necessarily a bad thing.
kylerush··on Busting the small startups can't A/B test myth
This is a completely fair point. Startups naturally will have to decide where the highest ROI is. It may not be a/b testing. However, they should know that a/b testing isn't necessarily off the table.
kylerush··on Busting the small startups can't A/B test myth
The point of the article is to help small startups understand that a/b testing is not off the table. Of course startups will have to decide where the highest ROI is themselves. Given a point in time, the highest ROI may not be a/b testing.
kylerush··on Busting the small startups can't A/B test myth
A/b testing doesn't have to mean extra work. If the small startup is going to redesign their homepage anyway (which a lot of them do), they might as well a/b test it.
kylerush··on Busting the small startups can't A/B test myth
You can click through on the Optimizely article to see the raw data: http://2nwl40151nra3yah86118nxtto0.wpengine.netdna-cdn.com/w...
kylerush··on Confidence.js – make sense of your A/B test results
Low traffic levels do not mean that you cannot do any meaningful testing. It means that you cannot reliably detect subtle effects in the conversion rate. Whereas Google may be able to reliably detect a 0.005% effect in the conversion rate, a small startup may only be able to reliably detect a 60% effect in the conversion rate. A small startup should arguably not be chasing after small effects in the conversion rate anyway.

Evan Miller has a great sample size calculator to illustrate this: http://www.evanmiller.org/ab-testing/sample-size.html

In the example above, Google's MDE is 0.005% and the small startup's MDE is 60%. If the small startup's baseline conversion rate is 5%, then with 95% confidence and 80% statistical power, they can reliably detect a 60% effect (positive or negative) with 1,968 visits.

It's not just the 60% increase that you should be mindful of. I bet you all small startups want to know if the change they made to their homepage decreases the conversion rate by 60% or more. With an a/b test, they can do that.

Another point to keep in mind is that you can learn a lot from a statistical tie (not enough data to conclude there is a difference). In fact, no matter what your traffic levels are, most of your experiments will be a statistical tie. It's important to learn what doesn't work just as it's important to know what does work. This can really help you with prioritization. In the example above, the small startup can use the statistical tie to draw the conclusion that some tasks in their product roadmap will result in less than a 60% effect in the conversion rate and can be prioritized as such.

But ultimately it will be quite difficult for the small startup to produce variations that have at least a 60% effect in the conversion rate, but it's not unheard of and again, you can learn from a statistical tie.

Just because you have low levels of traffic doesn't mean you can't learn anything from a/b testing.

kylerush··on Mozilla employees tell Brendan Eich he needs to “step down”
I actually consider it worse for an individual to have contributed to the marginalization of a group (in this case financially) of American citizens than to just have a documented instance of disparaging comments towards the group. In the former the individual is actively involved in marginalizing the group whereas with the latter the individual is stating an opinion. Your opinion matters, but what you do with your opinion matters even more.
kylerush··on Meet the Obama campaign's $250 million fundraising platform
We used Optimizely (https://www.optimizely.com/) to conduct most of our a/b testing. Optimizely works on the client level (browser), not the server level (CDN).

Optimizely is the best a/b testing software I've ever used. It's flexible and easy to use. I suggest you check it out.

kylerush··on Meet the Obama campaign's $250 million fundraising platform
The increase in performance that I mentioned is directly due to the performance increase. We built identical pages on both platforms and tested them against each other at the same moment in time.

The only difference between the two pages was the page load time. Because we removed all other variables (time, appearance, functionality, etc.) we can say that the 60% performance increase lifted conversions by 14%.

kylerush··on Meet the Obama campaign's $250 million fundraising platform
You tell me when and I'll be there:)
kylerush··on Meet the Obama campaign's $250 million fundraising platform
Ah! Sorry, misunderstood. In that case you are not wrong :)
kylerush··on Meet the Obama campaign's $250 million fundraising platform
If you think anyone on the campaign was being paid that handsomely you should look at our FEC reports. I took a 15% pay cut to work twice as many hours and so did all of my colleagues. The money went to where it was most needed and the proof of that is that we won the election.
kylerush··on Meet the Obama campaign's $250 million fundraising platform
Address is pretty standard for anyone accepting payments.

As for employer and occupation, we are legally required by the Federal Election Commission to do our best effort to gather the donor's employer and occupation. Doesn't have anything to do with taxes.

kylerush··on Meet the Obama campaign's $250 million fundraising platform
Yes. It was very simple. We just put the URL to the vendor hosted platform's POST endpoint in the donation form's action attribute so that if you didn't have JavaScript the form would POST like a normal form.
kylerush··on Meet the Obama campaign's $250 million fundraising platform
The API was quite simple. Essentially it just processed a donation. It had some other functions to it that involved saving a users payment information to their account as well.

Yes, the donation submission was an POST request executed with JavaScript (AJAX) from the user's browser.

Thanks for noticing the bug! I'll have to check that out.

kylerush··on Meet the Obama campaign's $250 million fundraising platform
Our Devops team configured Akamai so they would have specifics for you (@scottvdp).

With most CDNs you have what's called a pull zone that points to your origin server on a different domain. So when the user requests something like cdndomain.com/whateverpage the CDN looks for the file at pullzonedomain.com/whateverpage and then it serves what it gets back from the pull zone.

Akamai has more advanced configuration options than Amazon Cloudfront. One example of this is that our payment processors did not actually live on contribute.barackobama.com, only the static files did. So we used Akamai to setup a reverse proxy to POST the donations to a different domain. Without this reverse proxy it wouldn't have been possible because the same origin policy forbids cross domain, client-side POST requests.

kylerush··on Meet the Obama campaign's $250 million fundraising platform
Thanks for the interest! I am planning to write another blog post on all the UI optimization soon.
kylerush··on Meet the Obama campaign's $250 million fundraising platform
We had roughly 500 donation pages generated by Jekyll.
kylerush··on Ask HN: Who is Hiring? (August 2011)
Obama for America - Chicago, IL

Obama for America (OFA) is hiring many frontend developers to work on barackobama.com.

We are essentially a large startup that hit the ground running in April of this year. We need skilled frontend developers to help us create another groundbreaking information and mobilization site to help re-elect the President.

We're looking for developers with experience in HTML5, CSS3, JavaScript, jQuery, jQuery mobile, ExpressionEngine, social APIs, Git, A/B testing and anything else that is necessary to achieve our goal.

OFA is a very unique opportunity to work for the President on a high profile site that has tons of traffic. One of the many benefits is working with a very talented design team.

If you would like to have a part in helping the President strengthen the country and improve the lives of millions of Americans, contact Kyle Rush (@kylerush OR krush@barackobama.com). Job description/posting here: http://www.authenticjobs.com/jobs/9256/frontend-developer

← PreviousPage 2 of 2