Really just chroma + openai is all you need. Chroma makes this easy https://docs.trychroma.com/embeddings.
All you have to do is chunk... I'd start at a paragraph and experiment.
All you have to do is chunk... I'd start at a paragraph and experiment.
I would therefore suggest a better default for this, such as BERT+Qdrant.
It would be wonderful if there were a simpler (single file, SQLite or DuckDB like) database for vectors than the complex (and in some cases, unfortunately cloud-based) ones available now.
> It would be wonderful if there were a simpler (single file, SQLite or DuckDB like) database for vectors than the complex (and in some cases, unfortunately cloud-based) ones available now.
This is literally chroma!
pip install milvus