373 karma · joined May 28, 2013
And you are absolutely correct about the bar charts. Not exactly sure why I chose line but they have been replaced now.
The testing didn't use the built-in benchmark mechanisms but it did use the *testing.T mechanism. It's basically runs of different algorithms on the same set of pre-loaded integers. And then I recorded the time (ns) it took to run compression and decompression (time.Since(now).Nanoseconds() where now was time.Now()).
The caching effect should be pretty similar for all the algorithms since they work off the same LARGE slice of data, afaict.
I will update the article to reflect the testing methodology (may be a few days before I can get to it though.)
THanks
@bradfitz thanks for the comments and suggestion. I updated the code based on your suggestion (not allocate for every add/check) and the performance improved by 25+%!! Appreciate it!
I updated the blog and here's the gist that contains the latest results: https://gist.github.com/zhenjl/6515577
You are correct. In fact, I am allocating in both standard and partitioned Check() implementations, in the bits() func.
I can pre-allocate during New() and keep using the same slice without re-make.
Thanks for the suggestion! I will update and re-run.
I also need to go back and figure out the murmur3 implementation is not working.
In terms of rewriting a bunch of code, msgpack and bson both easily marshall/unmarshall in just a few lines of code so not a lot of rewriting. Libraries exist for all types of languages.
Maybe I need to update my criteria. Thx for the comment tho.
Type casting like this makes my brain hurt.
b = uint32(int64(b) * int64(c1) + int64(int8(v)))