Even 5 people is enough to significantly narrow the uncertainty (at least, as compared to the prior hypothesis that the true change in odds is somewhere from -1 to 1).
And like a good spam filter, you don't need to be perfect, you just need to be able to turn your initiatives into a forecast of future changes if implemented, and then measure what actually does change for those initiatives you do implement. Even simple linear regression models can be useful in tracking results compared to forecasts.
Most people would probably expect that the improvement is close enough to zero to be nearly indistinguishable from it. And this is why most enterprises don't invest much in reliability work like this, because it doesn't actually seem to move the needle for customer purchasing.