Startups: Your Users' Behavior Is Not A Random Variable

Excellent post!

And if you follow the latest online copywriting best practices, you'll know that creating 3-5 persona types for your website navigation is all that is required. Almost every one will follow one of those 3-5 pathways.

Sidenote about the test the article runs - re: adding text to a graphic tab...

In 1985, after finding that pretty but unlabeled icons confused customers, the Apple Computer Human Interface Group adopted the motto, "A word is worth a thousand pictures," And a descriptive word or phrase was added beneath all Macintosh icons.

So... I don't own a startup, have never done user interface testing and am generally unqualified to talk about how to design these things.

However, I'm not bad at statistics and logic, and I'm sorry, but the logic here is pretty bad.

The author is comparing a user (edit: really user behavior on a feature/icon that no one gets) to a fair coin. The conclusion he comes to...? They are not the same! Not exactly a lot of insight.

A much more fair comparison, in the example given, is comparing the user to a heavily biased coin. In this case, a coin that (almost) always comes up heads. With this coin, the convergence is much faster.

The (almost) fair coin is a better comparison to a feature that (almost) half of your users do/don't get.

So, if you try with 30 users, 14 like something and 16 don't, you need more data! But, if you try with 30 users, and 29 like it and 1 doesn't, you can stopy.

Great post, really wonder what the author's thoughts on A/B and Split Testing. Do you think it adds value?