Listen to users, but only 85% of the time

The most adamant feedback I’ve ever had from a user was for a feature that involved making a big solid block of colour on this particular page of a website flash constantly if a particular status was present in the data. I stood there pointing at a screen saying “are you really sure you want this…” (gesturing to how large it would be on their screen) “to flash bright red on and off every few seconds” …

We knew it was a bad idea, implemented it, and then pushed it to the preview environment, leaned back and waited, 5 minutes later the person comes up stairs and says “you have to change it”… and another and another keep saying it needs changing for the 15 minutes it takes us to turn it back off…

Sometimes users are definitely wrong and you absolutely know it… but sometimes, just sometimes… you might still want to listen to them in order to get them to actually listen to you and talk a little more deeply about how to solve their problems.

TL;DR: Data-driven design is a proven success factor that more and more digital businesses embrace. At the same time, academics and practitioners alike warn that when virtually everything must be tested and proven with numbers, that can stifle creativity and innovation. This article argues that Taleb’s Black Swan theory can solve this dilemma. It shows that online experimentation, and therefore digital design, are fat-tailed phenomena and, hence, prone to Black Swans. It introduces the notion of Black Swan designs — “crazy” designs that make sense only in hindsight — along with four specific criteria. To ensure incremental improvements and their potential for innovation, businesses should apply Taleb’s barbell strategy: Invest 85‒90% of resources into data-driven approaches and 10‒15% into potential Black Swans.