4,973 karma · joined October 20, 2014
I love chess, deadlifting, and Elden Ring.
Twitter: https://x.com/frankzliu
LinkedIn: https://www.linkedin.com/in/fzliu
How well would this work in a production setting, e.g. when searching over millions of PDFs on arxiv (soon to be tens of millions)? Follow-up: have you tried using a vector database such as Milvus as the key piece of underlying infrastructure to avoid having to implement deletes, failover, scaling, etc? https://zilliz.com/learn/what-is-vector-database
With that being said, match between Ian and Ding would also be incredibly entertaining. I look forward to it.
I recently built a similarity search application that recommends new Pinterest users channels to follow based on liked images using Milvus (https://github.com/milvus-io/milvus) as a backend. Similarity learning is a huge part of it, and I'm glad more and more tools like Quaterion are being released to help make this kind of tech ubiquitous.
This is why pure C projects can turn into an unmanageable behemoths. This is also why the best way to use C is to use it to implement core algorithms and data structures which can then be called by higher-level languages. Numpy/Scipy did this perfectly and now their use is now ubiquitous within the Python community.
Most software engineers I know who have a background in EE love C, simply because it maps very well to what a processor actually does during execution.