Imagine a scenario where option B does 10x better than option A during the morning hours but -2x worse the rest of the day. If you start the multi armed bandit in the morning it could converge to option B quickly and dominate the rest of the day even though it performs worse then.

Or in the above scenario option B performs a lot better than option A but only with the sale going, otherwise option B performs worse.