Hacker News Reader

A/B testing does not inherently lead you to good design. If you A/B test what people want based on their revealed user preference, you get gambling apps and doomscrolling.

"I want a steak dinner." Yeah, but based on usage patterns, here's some cigarettes and beer. /s

Even better is actual testing.

They put windows 95 in front of users, asked them to perform tasks, and, at the end of the test, asked them to shut down the computer. Almost everyone clicked on the start menu first, so that's where they put the shut down button.

It has become very clear over the past decade or so that you cannot make truly good software just going off A/B testing. It leads to something which is tolerable to everyone, but not actually good for anyone, and which taps into the self-destructive impulses of your users (doomscrolling, etc).

All of the pizza chains from my youth A/B tested their ingredients into a weird hill climbing problem. They are all worse tasting and more expensive but despite tasting very differently they are basically equal in how good they taste. You can't really say one is better than the other but they are definitely all worse than they were before.