Show HN: Elysa Local LLM Powdered Mental Health Assistant
play.google.com
play.google.com
This is going to tell someone to kill themselves eventually, what are the statistical guarantees that something like that isn't going to get you sued or worse, thrown in jail? Do you even know what the statistical chances are of it telling someone to engage in self harm? If so, how did you measure this? Do you even care? People are going to be sharing their most intimate secrets with this thing, what are your policies around data retention as it relates to the data that is sent to the AI provider, and how can you provide guarantees that data won't get exposed, either on your end, or the ai provider?
Tech masquerading as mental health should be illegal, particularly when it has a chat agent that will at some point hallucinate. Hope you can sleep well knowing any day you could wake up and see a message that your chat app drove someone to kill themselves or someone else.
This whole thing is irresponsible. I know you've put in a lot of work, but you did you stop to think about any of the rest of it?
Hallucination is not an if, but a when. You need to show some sort of statistical analysis that hallucinating is kept to a very low percentage.
We are programmers, we can't do everything, or know everything, we must reach out to and include domain level experts. Anything less is simply hubris doused in irresponsibility. I'm sure there are psychiatrists that would even assist you for free.
I could see many ideas from your app, as I've had similar ideas. But it all rests on the reliability of the llm.