Hacker News Reader

Comment by ceejayoz | Hacker News Reader

> I asked Google Gemini…

Why?

To understand how doctor's recommendations for in-patient care are treated in the health system of another developed country?

Why do you trust Gemini to do that?

Because LLMs are pretty good at summarizing facts from publicly available sources.

LLMs are notoriously good at getting even basic facts wrong.

Especially when the information ecosystem around the issue is... adversarial.

Source? In my experience, LLMs are especially valuable in that setting. Human generated content tries to put a spin on facts. But with LLMs, you can pull the facts out of such content while making your assumptions explicit.

For example, I could point you to this article talking about 12 month+ waiting lists for UK mental health treatment: https://www.rethink.org/news-and-stories/news-and-views/2025.... What is the organization’s spin? Am I cherry picking this article in comparison to others? I don’t know!

It seems like there’s a big advantage to an LLM synthesizing a result from a large corpus of articles that contain facts embedded within spin and narrative.

Great example!

I go and ask Gemini about the American equivalent, and get this:

> Roughly 40% of the U.S. population resides in federally designated Mental Health Professional Shortage Areas (HPSAs). The shortfall is severe: the national average wait time for care is 48 days, with over half of all U.S. adults with mental illness going completely untreated.

Is an 18 month wait worse than an infinite wait?

What's the average wait in the UK? Gemini says:

> Average Initial Wait: National data shows the average wait for a first appointment is around 3 to 4 weeks (approx. 22 days).

See, if you get satisfied with "UK mental health care sucks" answers, you don't ask the fairly obvious follow-ups and wind up confirming your incorrect priors.

(And to be clear, I don't trust Gemini on this, really. But if you do… why not ask it these questions too?)

I’m not making a point about UK wait times; we’re talking about LLMs as a research tool. I’m using a link I found through Google search as an example to show why LLMs are useful for such questions. As I said in my post, the problem with looking at articles to find an answer is that I don’t know if the data is cherry picked to push a narrative.

Your Gemini query actually proves my point. Average wait times for mental health care in the UK NHS are shorter than you’d think if you Google and search for articles. Because most articles are trying to peddle a narrative.

The fact that 50% of mental health issues go untreated in the U.S. is also interesting. Let’s do a similar query for the U.K.: https://share.gemini.google/69TASFO2sA3S

“Based on the most recent data from the Adult Psychiatric Morbidity Survey (APMS) for 2023/2024, approximately 52.3% of adults in England with a common mental health condition go untreated.”

So it looks like 50% of mental health issues go untreated in the UK as well. Doesn’t that show the value of using LLMs to pull out facts, instead of reading articles that wraps the facts up in spin?

> I’m not making a point about UK wait times; we’re talking about LLMs as a research tool.

And I'm noting that what and how you ask matter.

(As do the basic factual flubs they make. I use LLMs a lot on code! I don't trust LLMs on code as a result. Useful? Sure. But "I asked Gemini" really adds little to a conversation about something so complex as comparing healthcare between nations.)

> So it looks like 50% of mental health issues go untreated in the UK as well. Doesn’t that show the value of using LLMs to pull out facts, instead of reading articles that wraps the facts up in spin?

If you ignore the "common" qualifier in there, sure. Are you comparing apples to oranges?

Most people don't want to think any more than they absolutely have to.

Reply on news.ycombinator.com