I do not think you can just stop the test when you find something you think is significant

http://www.evanmiller.org/how-not-to-run-an-ab-test.html 'If instead of deciding ahead of time, “this experiment will collect exactly 1,000 observations,” you say, “we’ll run it until we see a significant difference,” all the reported significance levels become meaningless. '

Reading the stats.py it looks like you might be doing this.