Hacker News Reader

A/B testing can be done with feature flags. Gradual rollout, specific customer opt-in, and so on.

I guess you could do A/B testing with feature flags but honestly, it won't be any good. If you want a good answer if to go with A or B, you want to have many data points, not just from one specific customer.

Gradual rollout and specific customer opt-in to certain features can certainly be done with feature flags, I wouldn't call that A/B testing unless you're running experiments and reaching a conclusion that A is better than B (or vice-versa).

You can totally implement A/B tests on top of a feature-flag framework. Your “pass” function should be hashed on the User ID/Cookie/whatever and then you can distribute users into pass/fail. You should do this anyway for reproducibility. If you log whether they passed/failed and then have metrics, you compute experimental results.

Reply on news.ycombinator.com