Show HN: Run your own A/B testing back end on AWS Lambda and Redis HyperLogLog
github.com
github.com
Instead you should use a bloom filter plus a simple counter:
if not bloom_filter.might_contain(user_id):
count.incr()
bloom_filter.add(user_id)
A bloom filter can easily be tuned to have 1e-6 false positive rate (meaning a 1e-6 error rate in the counts). You just can't reliably do that with HLL.There's an experimental branch[1] I was working on[2] that uses Google BigQuery instead of Redis. I'd love to take a look at your bloom filter suggestion though.
As I mentioned, I'm eager to get collaborators to join the project. Any input is appreciated.
[0] http://antirez.com/news/75
[1] https://github.com/Alephbet/gimel/tree/bigquery
[2] http://blog.gingerlime.com/2016/a-scalable-analytics-backend...
conversions x (1+0.008) / [visitors x (1 - 0.008)
~ (conversions / visitors) (1 + 2 x 0.008 + ...)
~ (conversions / visitors) (1 + 0.016 + ...)
(It should actually be a bit less if you do the calculation carefully, probably 1.6/sqrt(2).) 12kb is also a pretty big filter, but I guess reasonable.It's definitely true that non-statistical errors are highly likely to drown this out. But why have extra error if you don't need it?
Any tips for implementing this with bloom filters using redis/python and how to reduce or even estimate error rates there?
I'm looking for contributors, so if you're interested - please get in touch[0].