Hacker News Reader

Comment by a2128 | Hacker News Reader

I feel like I've too often seen in products new (anti)features that are way too easy to accidentally click, and whenever I do accidentally click I just imagine it's increasing some statistics counter that's ultimately showing the product managers super high engagement, clearly meaning the users must love it to be using it all the time

View on HN

To your point, my company is doing some A/B tests and I insisted that we not just measure conversion ("it works") and additionally measure some metrics that would indicate that it works _well_. For example, if you have a carousel of products, and someone buys something from the carousel, then it "works," but it would work _better_ if the item they bought was the first thing on the carousel rather than the last. That indicates that we showed relevant products first, which is better than showing the relevant product last!

Sure, conversion would go up if it was in the first slot versus the last, but it takes effort (however little) to scroll through a carousel, so ensuring that we can measure the quality of the result and not just the quantity is really important.

This is one way I've tried to avoid the problem you describe. It's not enough that people can engage with the feature, but they need to engage with it meaningfully and in such a way that would encourage repeat behavior.

Another example of what you describe is that on our site if you search for "neon blue guitar", don't interact with the search results whatsoever, go back to the home page, click on a product on a carousel, and purchase _that_ product, it counts as a "successful search event," even though the search technically failed because they didn't interact with it in any meaningful way. To your point: PM is happy; user is not.

TL;DR it's really important to think through tests and how you measure success!

> but it would work _better_ if the item they bought was the first thing on the carousel rather than the last

Depends, does that increase overall sales? Or is it ‘better’ to make the customer ‘walk past’ the other items to get to the thing they want (the way supermarkets make you walk up the back of the shop to get to the milk), and maybe buy something else too?

Absolutely! It's important to have a hypothesis and test it, to OP's point!

One thing I always accidentally click is the animation-expanding Google results when I return back to the search results after visiting one of the result pages, while trying to quickly visit the next result’s page.

Then in the code there’s a bug that over/under reports those clicks (because ui is not procedural code that lends itself to straightforward metrics) and i think this could explain Spotify’s product decisions.

Several years ago Google implemented this A/B feature in which you had to choose one image or another. Do you remember that one? Ofcourse I always chose the wrong one ;) It didn't last long.

Reply on news.ycombinator.com