Hacker News Reader

A/B test: Do users who have the feature enabled get you better KPIs or not.

How do you track that exactly? Any chance you can share a concrete example?

Randomly, take 10%-25% of your users, show them the feature and see if they do more of what you want them to do.

If you're blogging platform, that would be writing more blogs, or whatever metric that guides your business growth.

If yes, then roll-it out to 50%, confirm that the metrics are moving to the same direction and by how much. You should be able to answer the question: "users who have this feature do $X more/less/faster/etc.."

You can implement this all in house or use one of the tens of A/B testing SaaS out-there.

Got it, makes sense. Appreciate the concrete example! As for the event tracking data, do you use a tracking plan, like Segment Protocols?

I'm not familiar with Segment, but you can't go wrong with launchdarkly[0] even though they get a bit expensive recently.

[0] https://launchdarkly.com/

Cheers!

Reply on news.ycombinator.com