Comment by olivierduval | Hacker News Reader

Comment by olivierduval | Hacker News Reader

Well... I think that it's a bit too early to think about this:

1) chatbot are either based on a search engine (so with limited answers) or AI-based (with sometimes totally inaccurate "understanding" or "answers")

2) knowledgebases (and other kindof FAQ) are still really an open problem: wikis are a way to structure informal knowledge but need A LOT of discipline and human... FAQ are generally limited to the 10 most questions

In the end, the best way to provide knowledges right now is the old "website search engine"... but nothing more. And AI has already shown that - even if it LOOK good at first sight - it's not GOOD ENOUGH to be used (wrong answer looking "right" for example)

Moreover: a website embed a structure, so you know what you can search or how to formulate it even if you dont know what you are searching exactly... a chatbot requires you to ask the right question to get the answer you seek, and that's far more challenging for the user !

As an example: try to ask a tax chatbot about how to get family support... and compare it with looking at the website structure (that clearly show that you're not at the right place)

The chatbot experience you describe sounds like pre ChatGPT. The big change in performance showcased by ChatGPT is what makes me think things are about to take off. It’s really a whole new ball game.

Actually, ChatGPT is exactly the reason why I don't see chatbot taking over the world of knowledgebases !!!!

Right now, for what I read, ChatGPT is able to "synthetize" answers that are just false ! And even if it's right 95% of time, you just can't afford to have 5% answer being false... Just imagine what would be the consequences if ChatGPT "decides" to discount your product or say that a non-existing functionality is implemented...

ChatGPT might be some kind of answer IF it's the beginning of a pipeline that is followed by some kind of "reality check". But right now, I wouldn't trust this technology to replace some "certified and validated" information on a website representing a product or a company

Hallucinations are a real problem right now, and that needs to be solved. But we don’t need the tech to be perfect in order for it to have a massive impact.

One question is how far off we are from a model that is reliable enough to autonomously handle hard business transactions. But an entirely different question is how much better could many user experiences be that don’t involve an automated culmination of a hard transaction.

I think of you differentiate this way between transactions with a hard impact (ie charging a customer or shipping a product) vs. informational transactions, you can allow for a lot more to happen on a much faster timeline.

> Hallucinations are a real problem right now, and that needs to be solved. But we don’t need the tech to be perfect in order for it to have a massive impact.

As long as the use cases are considered properly.

Consider I hit a pet emergency website that is powered by ChatGPT. I ask it what to do because my dog has just had an encounter with a specific poisonous animal. It gives me advice based on a similar, but slightly different situation and my dog dies.

Alternatively say a telecommunications vendor (or for that matter, many SaaS vendors) implement it instead of detailed product docs. It gives me wrong advice and I misconfigure a router. As a result, an emergency call is mis-routed and someone dies.

AI chat isn't even remotely ready for these scenarios yet and there are still plenty of critical infrastructure services that rely on things like email. Therefore even your basic webmail SaaS isn't a suitable target for this type of tech.

Where could you genuinely deploy it safely?

Well, you need one level of readiness for life and death apps and a whole other level for entertainment apps and then there’s everything in between.

One can debate just how much of the market falls into the safer range of the spectrum, but the key question may be how far we are in time from much more reliable performance and not how far the current performance falls below some reliability cutoff.

For the sake of this discussion, I think how far we are in time is the more compelling question. And, I don’t know that we are nearly so “remote” in this sense as suggested by the phrase “isn’t even remotely ready.”

Unless there is specific reason to believe otherwise, I’d expect such improvements could appear in very short order given the recent pace of model improvements in other ways.

Reply on news.ycombinator.com