Excellent post! Great details!
But I have a question: You mean that vectorized execution helps make the best of modern CPU hardware characteristics. Have you tested the performance of these vectorized functions for different hardware architectures and different generations of CPUs?
We tested on some machines, but didn't perform comparion tests for CPUs of different generations. We used perf to compare vectorized and non-vectorized programs, and found that IPC and the cache hit rate for vectorized programs improved. Because of limitations of space and time, I can't give such details in this post.