Lesson from Madlibs Signup Fad: Do Your Own Tests
kalzumeus.com
kalzumeus.com
As far as design decisions like these go, the data doesn't say anything on an absolute scale about the merits of one form versus the other, but it does tell us which is more effective for a particular audience in a particular context.
When you are done reading the Madlibs, it should read like a sentence. What he did was surround regular text fields with strange sentences. When you are done filling it out, you can't just read it like a regular sentence.
It should have been:
I would like to make bingo cards! Please save them for me under the name [ Your Name ], and and my email address is [ Email address] in case you need to contact me. But please [do / don't] contact me twice with hints, and [do / don't] send me a newsletter.
See, when you are done, you can read it like a sentence.
All he did was replace "Username" with longer wording. It didn't seem "fun" the way you did it, just odd and off-putting.
I am skeptical that "looks pretty" drives conversions. I think customers pretty much don't care, based on "ugly" versions winning in many, many of my tests. Remember: the customer doesn't get to see the pretty alternative laid out side by side with the ugly alternative.
Example: form submit buttons in your Internet browser of choice look fairly pretty, right? (Or at least comfortingly familiar.) I would not describe http://www.bingocardcreator.com/images/bcc.net/next-step.png as pretty -- it is what I could throw together in ten minutes. But it clocks "pretty" buttons with the same text on it.
You're welcome to try it yourself if you think pixel-perfect alignment will make a difference. Please post results.
In the Huffduffer example the keywords are exposed, the fields look clear and spacing is just the way it feels nice. There's also a rhythm - keyword, field, keyword, field... even though it's not a standard form - you just know what to do with it. LukeW's implementation is not as pleasing, but still looks good, because there's some whitespace and no clutter.
I don't think you actually compared good standard form to good madlibs form. You compared it to a hacky, badly implemented madlibs and wrote off the whole idea ("Anyhow: test ended, not touching the madlibs idea again").
Not that it matters -- the data matter -- but I can certainly rationalize why such a button would be more effective than a standard form submit button: My peripheral vision has seen a hundred thousand form submit buttons by now. I'm not quite as blind to them as I am to standard-sized non-animated web ads, but it's close.
And it seems to me that the problem with being a designer is that it's physically impossible to design a web form in less time than the average user will spend looking at it. You will necessarily spend orders of magnitude more time looking at your forms than a visitor will: Visitors don't look at forms for fun. They click through them fast, or they click away from them even faster. If I had to stare at that button for an hour, as opposed to simply finding it and clicking it as fast as I could, I'd design it differently. Which might very well screw it up.
The madlibs form is just too wordy. Whereas the username/password field pair is not only less wordy, it's an idiom: I've seen those fields, in that arrangement and that order, thousands of times over the last decade. I barely even have to read the captions at all.
Not that this hypothesis, or any of the others, matters. The point of this article isn't to design a better form. It's to show us how to design a better form.
You provided us with reason to believe this (albeit simple) change will not provide us with results.
Please, don't do this.
For example, in the original madlibs test, where seven things changed, we want to attribute the change to the madlib element (which was sexy and creative) rather than to e.g. the text on the conversion button changing. I've gotten 20% lifts out of "stupid little things" like that before, and seen good writeups of the same from other people. But e.g. designers do not want to hear that the thing they spend 98% of their time on is essentially meaningless and the elements of the page which they scrutinize least carefully -- like microcopy or button text or (in one memorable test) which way the stock photo model is facing dominate the measurable results. This is like how doctors do not want to hear that the difference in clinical outcomes due to professional experience is totally dwarfed by the difference in clinical outcomes due to following checklists. (On the plus side, designers don't kill anybody for their professional pride.)
In general, I think the point of A/B testing is that you should ruthlessly soulcrush what you want with what the data actually tells you. I respect HN folks enormously, but I'm not hearing "I tested it and it worked" -- I'm hearing "I've got a great theory on why your data does not conform to reality. Reality is, of course, what I predict it to be."
The majority of websites have used the standard sign up form, and it's known and recongized by most users. It's risky to to go with something like this that could have a strong negative impact if you do it wrong.
Both forms have a 50/50 chance of having the guest link on them, and guests don't count as trial conversions (because I have several months of stats that suggests they give me money infrequently and they cause more support issues than all trialers and paying customers put together). The thrust of that A/B test is to see whether aggregate sales go down if the guests option is taken away. If they don't, I will drop it.