We stopped doing A/B tests after I insisted that they all be done as A/A/B tests. Suddenly the "clear winners" weren't so clear after all. It confused and frustrated the marketing department so much that it was decided just to stop doing them all together.

The reason I wanted this type of test was because it was a waste of time testing shades of blue or two headlines that only differed by 2 words. The test variants were never radical enough to see any kind of significant uplift. Then after 5-10 tests the design starts to suffer by wandering down some weird path that nobody would consciously design from the outset. But the series of test "winners" made things go off in wild directions.

I still think there is some value in A/B testing (A/A/B only, if I'm honest). But in a small team, it's a waste of time.