Hacker News Reader

Comment by PragmaticPulp | Hacker News Reader

I have a friend who has been going through some medical issues. He’s been using ChatGPT aggressively.

The one thing I can say is that he’s very good at getting ChatGPT to tell him whatever he wants to hear. Massaging prompts and providing different cues until the response confirms whatever theory he wanted it to confirm.

He’s currently convinced that various alternative medicine supplements will cure him, and sure enough he has ChatGPT logs confirming his theories. The exact alternative medicine changes by the week and he quickly forgets about last week’s cure when it doesn’t work.

He loves ChatGPT, but it’s clearly just a mirror for whatever theory he wants someone (or something) to confirm. Studies like this one make him think it’s more of a ground truth source of information that his doctors just aren’t smart enough to know.

View on HN

That type of test (have someone pushing ChatGPT to say that vitamins are good for you) would be more relevant than testing an LLM on its ability to recognize symptom keywords and match diagnostics, something that everyone (including their harshest critics) readily admit they can do.

The “MedMD problem” is so classic that a comment next to this one doesn’t have to explain why you always end up with cancer. And the supplement industry pushing PragmaticPulp’s friends’ delusion isn’t exactly discrete either.

All that, while every Western nation has record levels of low physical activity and social isolation: there is a real mismatch between medical advice to civilians and what those tools offer. A convenient, accessible medical dictionary is great, and I have no doubt it can process even more specialized knowledge, given enough strong-arming Epic (the de-facto monopoly on doctor’s note software) and a sprinkle of data engineering. However, that’s not the most apparent issue that this tool has. I’m concerned that OpenAI isn’t more focused on the bigger problem.

When I want ChatGPT to give me information, I'm always very careful about how I ask the question. ChatGPT is (usually) very eager to confirm your views, so you can't let it know what your views are.

It still makes mistakes, of course, but I'd say it's right more often than it's wrong, and more importantly the answers provides a starting point for learning more.

Even in the absence of AI, what's stopping this friend from going to a quack 'natural therapist'? If Steve Jobs can be fooled, what about an ordinary person?

ChatGPT is motivated by sycophancy, humans are motivated by profit. I would say the latter is much more dangerous than the former when it comes to medicine.

Remember, totally legit doctors were peddling opioid pills to patients in the name of "Treating undertreated pain". Profit distorts human minds very easily.

AI doesn't need to be perfect in every situation, just much better than the status quo.

> Even in the absence of AI, what's stopping this friend from going to a quack 'natural therapist'?

It takes a lot of time, effort, and money to shop around to numerous doctors until you can find someone who will tell you exactly what you want to hear about your alternative medicine theories. He tried, but was always complaining about the doctors/naturopaths.

But at home, all he had to do was open ChatGPT and spend a few days learning how to prompt it to provide the answers he wanted.

The situations are entirely different.

> just much better than the status quo

The status quo being regular doctors? Because that's the status quo, professional doctors.

Having a bot that you can cue and prime towards something is very dangerous in the medical scene.

> Remember, totally legit doctors were peddling opioid pills to patients in the name of "Treating undertreated pain". Profit distorts human minds very easily.

If ChatGPT was trained on those same data that these legit doctors were taught, it would lead to the same diagnosis.

I used to play drinking game: type generic symptoms into WebMD, and every time it says “cancer” take a shot.

Got pretty drunk pretty fast.

This kind of meme is really just pro-medical establishment to a fault. If you have a discerning mind and spend more than a few hours with tools like UpToDate you can make similar diagnostic conclusions that even specialists can make. The field is overglamorized when in reality the majority of what theyre doing is simple application of diagnostic algorithms.

Reply on news.ycombinator.com