861 karma · joined May 22, 2021
Creator of txtai (https://github.com/neuml/txtai)
So ultimately while good intentioned, you end up limiting how many people can use what you've built.
When it comes to a startup or company project though, it also depends on how much time you spend on something that isn't the core problem.
I'm of the opinion that what I build, I'm willing to share it and let others use it as they see fit even if it's not to my advantage.
https://github.com/neuml/txtai/blob/master/examples/78_Acces...
Multi-vector models
Multi-vector models are more expressive through the use of multiple vectors per sequence combined with the MaxSim operator [Khattab and Zaharia, 2020]. These models show promise on the LIMIT dataset, with scores greatly above the single-vector models despite using a smaller backbone (ModernBERT, Warner et al. [2024]). However, these models are not generally used for instruction-following or reasoning-based tasks, leaving it an open question to how well multi-vector techniques will transfer to these more advanced tasks.
Sparse models
Sparse models (both lexical and neural versions) can be thought of as single vector models but with very high dimensionality. This dimensionality helps BM25 avoid the problems of the neural embedding models as seen in Figure 3. Since the of their vectors is high, they can scale to many more combinations than their dense vector counterparts. However, it is less clear how to apply sparse models to instruction-following and reasoning-based tasks where there is no lexical or even paraphrase-like overlap. We leave this direction to future work.
In other words, it says that both multi-vector (i.e. late interaction) and sparse models hold promise.
https://github.com/neuml/txtai/blob/master/examples/58_Advan...
There are plenty of lightweight retrieval options that don't require a separate vector database (I'm the author of txtai [https://github.com/neuml/txtai], which is one of them).
It can be as simple this in Python: you pass an index operation a data generator and save the index to a local folder. Then use that for RAG.
Basically, a lot of people who are experts are being told this story and they think they are the only one who doesn't get it.
There are plenty of gains to be had with AI/LLMs but just not in the way it's typically marketed.