Choosing the right approach to building your own AI assistant

dashworks.ai

6 points·techmarketer40··3 comments

The article mentions "human-in-the-loop" training: how does this work in practice? How much effort is required from the company's staff to train and maintain the assistant's accuracy?

(author here) This really depends on what you're trying to functionally achieve, the organisation of your knowledge base and the fidelity you're looking for.

Let's take the example of "where should the assistant look for information on topic X", the absolute minimums would be to identify possible topics & the hierarchy of places you could look.

From the product engineering POV for the build your own path, for a well defined, limited search space this should be easily doable by a single engineer in a few days to a week. As this scales out to an entire company's knowledge base this quickly becomes a quarters long project for a small ML team to build the ongoing training jobs, data pipelines & monitoring tools required to make it robust.

From the POV of users, we designed our system to give our users the option to provide as much or as little feedback as they like. We can go quite far with upvotes/downvotes on whole answers, but we also accept per reference votes & full natural language feedback. We're still working on even deeper feedback mechanisms for power users & admins but we've typically seen the vast majority of users engaging in per answer voting & then exponentially smaller groups in the more detailed mechanisms.

Good analysis of factors to consider when building a vector DB-based RAG AI assistant.

Reply on news.ycombinator.com