Hacker News Reader

Comment by turnsout | Hacker News Reader

Are vector databases like Chroma already doing some of this? If not, it could be a huge win for RAG workflows!

SimSIMD is already used in USearch, ClickHouse, and Lantern. In other solutions SIMD is generally limited to pre-2015 Assembly instructions. So yes, there a lot of RAG workflows to accelerate :)

the sad part is that they are the ones ripping the benefits of this tech by capturing that sweet VC money

I’m curious if pgvector is using it. It’s actually pretty fast for us without even indexing.

As far as I'm aware pg_vector just uses the compiler's autovectorization on float32. I think specifically for Euclidean distance you won't really beat GCC (the README even admits it: "GCC handles single-precision float but might not be the best choice for int8 and _Float16 arrays, which has been part of the C language since 2011.")

Yes, the improvements for float32 aren't very dramatic, but it can be 3x NumPy/SciPy: https://ashvardanian.com/posts/simsimd-faster-scipy/

C11 is also a bit tricky, as its support is optional, as far as I remember.

I also notice you use FMA in your AVX2 L2 distance calculation. I don't think pg_vector enables that, so SimSIMD might be slightly faster.

Also interesting that it beats NumPy/SciPy by so much! I wonder what they're doing..

Not using SIMD at all as it seems. Thus relying on auto-vectorization of I'm not reading it indirectly.

https://github.com/scipy/scipy/blob/main/scipy/spatial/src/d...

It doesn’t support f16 or i8 at all yet (at least not in the released version).

Reply on news.ycombinator.com