Hacker News Reader

Comment by tosser1234 | Hacker News Reader

I worked on this project and there were a lot of issues. Two of the biggest were:

* Whatever the quality of the technology (which I personally never saw as that compelling) was wrapped up in terribly written research code, making it practically impossible to setup and use.

* The Jeopardy demo was made possible by the existence of a marked-up source of general knowledge (Wikipedia), a ready-made bank of questions and answers from past shows (j-archive.org), and the fact that practically anyone had the ability to curate more Q&A pairs. This is almost totally different than the medical use case where the knowledge is wrapped up in proprietary textbooks and papers and the only people able to curate training data are medical professionals.

View on HN

That's not really true. The UMLS has a large graph of marked up medical domain knowledge that can be used. It's not as specific as one might want for developing an AI autodoc, but it's quite a bit better than what is available in most fields. It's actually quite similar to what one can derive from Wikipedia.

Unfortunately UMLS and Wikipedia are not good enough for drug information. Structuring that data requires a fine balance of tooling (with some NLP) and a good team of biocurators (doctors, pharmacists, pharmacologists and domain experts).

Quora has a nice set of answers about why machine learning isn't used so much in medical:

https://www.quora.com/Why-isn%E2%80%99t-machine-learning-mor...

Skip to Jae Won Joe's answer to see a case study with a patient showing the interactive process. Then, especially look at ending questions about missing or deceptive data. Seems like their good diagnoses comes from a combination of domain data and expertly reading people in front of them. Machines suck at the part I emphasized. The data sets might not reflect a lot of stuff like that, too. Who knows.

This is a great run-down of the realizations that a lot of the engineering team came to and what I was trying to touch on in my reply above. There is just so much information & context within a doctor's head and so much that is going on during an examination that looks simple to our eyes. Even in the idealized case of having accurate, structured patient data and focusing only on a simple disease, it's a challenge to diagnose and prescribe a correct treatment. Even a system that can accept the text of a patient's medical record and successfully pull out relevant details would be challenge.

I think this article addresses the half the medical landscape where the problems are hard and computers are worse than humans at solving them. The other half is where the problems are "easy". Practices where Doctors spend ~5 minutes with each patient doing physicals and treating minor ailments. In that half doctors can already easily handle the problems so there's no real value add for a computer.

It would take a paradigm shift before AI became useful. Going to watson.com and getting a prescription for antibiotics would be useful and technically feasible. So would augmenting lower trained people to be able to deliver more care. Neither are possible legally and it's not exactly a field you can get away with disregarding the law.

Direct link to the answer in question: https://qr.ae/TWT7Ee

Having that taxonomy is definitely a great starting point for the field, but just to build a model to extract patient data from the text of a medical record you'd need to:

* Procure patient medical files. This rabbit hole includes things like HIPPA compliance and sufficiently anonymizing the files. Also, hospitals consider 3 years of information on 100 patients to be a lot of data.

* Have trained medical professionals annotate the files. Good luck convincing doctors to do this. Also good luck in determining what to do if the handful of doctors you're granted a few hours with a week can't agree on annotations.

* Actually produce a model that is reasonably good at pulling information out of patient charts,

Even after all the above, you still only have a solution that is giving you structured data, which is something that other EMRs can provide w/o the problems of accuracy & completeness of NLP process. There is still the process of diagnosis and treatment and also the fact that this is all a loop ('active learning').

Reply on news.ycombinator.com