redskyluan

Although LLMs are impressive, we should keep an eye on the potential risks. They have the ability to generate hallucinations and should be monitored closely. It's not just the tech industry that's worried, governments all over the world are concerned too.

Totally agree with that. 8G might be too much to make the open source product popular.

I would say 4G may more sense to me, I know how much engineering effort it requires though. LOL

OSSChat is a ChatGPT and Vector database enhanced chatbot about open source software. Now we support AutoGPT!

You are right, most LLM queries won't be exact match, so you will need semantic search for hit those similar but not same questions

We build a cache for cache for ChatGPT, see how it can save your money and time when build application on top of LLMs

what about the https://huggingface.co/facebook/opt-66b?

I thought the opt series can be used in production

do you guys think it is good idea to have a cache layer on top of LLMs?

what will be the target user of this service?

I really like this idea. did you plan to add more oss projects into it?

would you please update the milvus score in your benchmark with the latest milvus, thanks

did you try to serve billion level vectors, on both projects you mentioned? Then you can define what is overengineered.

And what about ScaNN and GPU index? the flexibility to support multi index type is a actually a big plus

We should probably try to implement a PDF search demo on top of Milvus.. LOL

See https://milvus.io/

FAISS is simply a library. You will need a system Like Milvus for huge data amount

cool idea, what skillset are you guys looking for?