Boosting Vector Search Speed by 40% with AVX512 on Emerald Rapids

We explain how we attained a ~40% QPS speed up at 90% Recall in Weaviate running on Intel’s new Xeon Processor, Emerald Rapids.

We do a technical deep dive into different implementations for vector distance calculations and optimizations enabled by Intel’s new 5th Gen Xeon Processor - Emerald Rapids, parallelization techniques using SIMD with AVX-256 and AVX-512 instruction sets, loop unrolling and compiler optimizations by transpiling C to Go assembly.

nice, inference is still primarily a compute workload.