For an A/A/B test are you taking three samples (instead of two), and two of the three get shown the same thing (A)? Then you only consider the results for the B group if the two A groups show the same behavior?
If A' and B both statistically differ from A, then you have a problem because you're not testing what you think you are testing, regardless of what your naive A/B test's p-value would have indicated.
This helps to show the effect of a low sample from a non-uniform distribution.
A lot of people (me included) think they know statistics, but they don't.
The blogpost in OP also tries to explain the same thing - you shouldn't do statistics without understanding what it is your doing.