Comment by huac | Hacker News Reader

Top New Best Ask Show Jobs

Comment by huac | Hacker News Reader

Parent Full thread

I have gotten 10x speedups with SIMD on modern hardware - goes from fast to blazing. https://github.com/stillmatic/gollum/blob/main/vectorstore.g...

FWIW - Weaviate's pure Go implementation of cosine similarity is probably as fast as you can possibly get in pure go, without SIMD -- https://github.com/weaviate/weaviate/blob/2f44dbee52196656bd...

Weaviate actually does use SIMD for AMD64 with Go assembly (cosine distance is just normalized dot product) https://github.com/weaviate/weaviate/blob/master/adapters/re...

Yep, we'd go with what they did, except OpenAI embedding vectors are already normalized, so you can just do the dot product.

Do you need anything special to run a Rust program with SIMD enabled?

Reply on news.ycombinator.com