HNHacker News
TopNewBestAskShowJobs

rvrs

249 karma · joined August 15, 2023

empathetic wifi
submissionscomments
rvrs··on Do we think about vector storage wrong?
This is tangential, but another issue I've had with existing vector databases is write throughput. Weaviate and Pinecone build an index immediately after the creation of a "table" (or rather, the analogue thereof) and this makes write throughput abysmal at times.

We have around 2 million 768-length embeddings + metadata -- writing too fast to Weaviate quickly results in OOMs. We circumvented this by very slowly trickling our data through to Weaviate, but this causes many-hours-long writes.

I'm experimenting with dumping our data into postgres and building an hnsw index with pgvector once all writes are done. This has reduced our time to maybe 2 hours at most (the bulk of that being index building time, of course).

rvrs··on ThirdAI Uses Ray for Parallel Training of Billion-Parameter NN on Commodity CPUs
IIUC Anyscale is managed Ray, and also works on Ray full-time. However Ray is open source and you can run it yourself. It's fairly easy to get started with, too.
rvrs··on Ask HN: How to come up with a useful, coding hobby project?
Not everybody wants or needs to be an entrepreneur
rvrs··on Show HN: Llama2 Embeddings FastAPI Server
What's wrong with just using Torchserve[1]? We've been using it to serve embedding models in production.

[1] https://pytorch.org/serve/

← PreviousPage 3 of 3