Comment by Aurornis | Hacker News Reader

Comment by Aurornis | Hacker News Reader

Parent Full thread

> I really don't buy that this person's AI tool did anything here.

The author fed their medical records to the tool and then, as far as I can tell, the tool included autoimmune disease as a possible explanation.

Every comment that I saw asking for clarification received a vague response. The closest I can see is that the LLM and rheumatologist both proposed “an autoimmune condition” around a similar time, though the answer about which came first is also unclear.

The catch is that for the author to even have a rheumatologist referral to begin with, some previous combination of directors would have had to have suspected an autoimmune possibility and created some documentation to that effect. Those documents were supposedly fed into the tool, which obviously would have surfaced some of the data and possibilities that led to a rheumatologist referral.

So I agree. The way this tool is being presented as diagnosing a mystery illness by itself feels very misleading. I saw this spammed across multiple subreddits last week when the author was trying to promote it. I think maybe they set out trying to get attention on their AI side project on their GitHub profile, but got a little carried away with the angle that it was diagnosing medical mysteries.

> to even have a rheumatologist referral to begin with, some previous combination of [doctors] would have had to have suspected an autoimmune possibility and created some documentation to that effect

This is a good point. Even when a diagnosis is obvious, a doctor may not say it outright and instead make a referral to a specialist because it’s not their specialty to diagnose that condition. This is a good thing because they’re not that specialist so they could be wrong about it, and if they are right, then they should make that referral anyway - and end result is the same i.e. a referral.

I don't see the harm in something like this. If you were to train your own LLM specifically on medicine & medical data, then show it your lifetime history of doctors visits, blood work, what drugs you're taking, have taken, symptoms and etc etc etc there's a decent probability it could give some perspective you might not be able to get elsewhere - especially in the broken US healthcare system. Having access to multiple specialists in many different fields is something a large % of the population simply doesn't have access to for a variety of reasons.

I think the issue is people might assume it's going to spit out your exact diagnosis 100% of the time right off the bat. That's obviously not true, but that's not what I got from reading the description of the project. It sounds more like a potential useful tool for folks to have as an additional option, and then to bring that information to a doctor to see if it pans out.

Regardless, one extremely helpful thing I got out of reading the reddit thread was the mention of an unrelated project called Fasten, which will apparently allow you to request all your medical records from many sources and then self host them:

Fasten is an open-source, self-hosted, personal/family electronic medical record aggregator, designed to integrate with 100,000's of insurances/hospitals/clinics

https://github.com/fastenhealth/fasten-onprem

> If you were to train your own LLM specifically on medicine & medical data

That's a massive "if" that the rest of your argument hinges on. You can't train your own LLM on your own data. You can fine tune it's responses to be more in line with the format or style you want. But this is usually used as a cost saving exercise when you are feeding s big system or inital prompt for lots of queries.

> there's a decent probability it could give some perspective you might not be able to get elsewhere

There's also a decent probability that it will give a wildly incorrect perspective that influences how you approach your possible disease in an unproductive way.

The author has no idea how accurate the tool is. They list no effectiveness measures. They just talk about the cool things it does (again, without accuracy) and their subjective experience with it.

It's the same pattern with LLM based tools we see so often: someone gets lucky with a useful result that no one else has done yet, they assume it will work just as well for all inputs, they build an app around it. You _have_ to measure effectiveness across a wide variety and quantity of inputs because of how inconsistent LLMs still are.

> I don't see the harm in something like this.

Incorrect self diagnosis is a big problem for doctors. Patients will show up convinced they have one condition and reject their doctors’ attempts to explore other explanations.

It’s a big problem right now with TikTok self diagnoses. People view a lot of TikToks and think they know better than their doctor for themselves or, worse, their kids.

The easiest example is when people decide they have a bacterial infection when they really have a common cold. They will shop around until they find a doctor who is a pushover and will prescribe antibiotics on demand. This is a big problem right now with antibiotic overprescription.

The problem with this specific post, though, is that it’s misleading. The author tries to present this tool as if it did a better job than $100K of medical testing and 5 years of doctors, but when you read the details it’s clear that the results from all of those doctors and medical testing were put into the tool.

Many of the comments here and on Reddit are missing this key point: The tool was mirroring the diagnosis from the doctors because it was given the medical records from those doctors, not because it was diagnosing the symptoms from a clean slate.

Reply on news.ycombinator.com