Hao
Hao
Question I have for you is that one of the biggest problems with cancer diagnoses is false positives: "Yes, there is something on your scan, we're not sure what it is, so we'll biopsy it." Biopsy is not a 0-risk procedure, and it can cause a lot of worry and pain, so it's not something to be taken lightly. Also, there are many cases of "OK, it's probably cancer, but the cure may be worse than the disease." This is the classic problem with detecting prostate cancer at an advanced age - it's very likely/probable something else will kill you before the cancer does.
How does your software deal with this issue? I'd be worried if, as you put it, people in a remote area with limited access to experienced radiologists, were given access to this and it came back with "Fairly decent chance of cancer" - what do they do then?
Hao
yes, I know mammo AI products are on the market- those that were approved followed a collection of regulations that I think you are not.
It doesn’t really change anything.
Even if you have done everything you are supposed to do in the process, at the end of the day you are looking at ROC curves or equivalent and trying to understand sensitivity vs. specificity trade offs, and often you have some sort of (indirect) parameters than can move to different points of that tradeoff.
This is quite critical in deployment, if you are screening you usually want something different than in diagnosis; as alluded to elsewhere if you raise the work-up rate too much you definitely risk killing more people from biopsy complications than you help with higher sensitivity (it's more complicated than that in practice)
Nothing abotu what I'm saying is uncharitable- in a sense, it's charitable because I'm helping warn a person who is going down a dangerous path to consult more with experts in this field.
What you did was assume that they were ignorant in potentially dangerous ways, and assert they should do something different.
I don't think a reading of the comments/responses (at least at your time of posting) really supported that assumption, especially considering the limitations of the medium. Hence my reply to you, while also detailing the trade offs a tiny bit.
Please be assured that I put a fair amount of thought into this - for example, I used to do due diligence for a VC firm evaluating proposals like this all the time and we had to reject most of them because the founders didn't understand the basic rules of deploying medical technology in highly regulated environments like the US.
Based on my interactions with the author in the various parts this post, I continue to conclude this individual is lacking core knowledge and wisdom required to execute a project like this successfully at scale.
Anyway and least at they time you posted (since then there were more interaction) I didn't find the same information nearly enough to dismiss their competence out of hand.
What I didn't totally grasp from the article is this is a company https://www.reddit.com/r/MedicalPhysics/comments/t5u2c9/intr...
So this isn't an amateur programmer, it's a person who got a phd in nuclear engineering and radiological sciences, was a scientist at bruker, has some experience with health systems, and then became a serial entrepeneur with a small company that has some funding. BTW, people who have the job title "Director" are normally fairly senior (old), as well.
he worked successively as Director of R&D in imaging at Bruker and Siemens, directing product development in the imaging detectors.
Is where I got Siemens from.Out of curiosity, how are you handling the data access and labelling issues here? I suspect that's the key issue that has limited the performance of the commercial offerings (hardly limited to this problem or this space).
OTOH in terms of real impact, properly leveraging a more modestly successful algorithm will probably help more people than getting a few more %. With the (strong) caveat that in a space like this you really have to look at work-up rate and balance risks.
> to at least get a second opinion on their mammogram.
for me makes a lot of sense, even in developed countries where you get a result but want extra assurances.
It would be interesting to know (assuming you have the data, even anecdotally) if the second opinions using this overturned professional ones and from those how many were corrected an original false negative mistake.
Screening breast mammo has an occurrence rate problme. Something like less than 10 in 1000 studies will require further review; this means in practice as a radiologist you look at a lot of negative films before seeing a TP. It also means a typical read is done fast. Seconds-to-small minutes.
This results in a couple of things. Reader variability based on experience/throughput, and false negatives. There were some double reader studies that caught something like 15% (going from memory here) of FN - but nobody can affort to have two radiologists read everything.
So the profession is already conceptually used to the idea of using an algorithm as a "second read" and reconsidering. Typically this won't "overturn" anything here but rather say 'hey have another look', but the decision to proceed or not is still the clinicians. Having a positive from the algorithm makes them review carefully, but you have to watch the FP rate here or nobody would get anything else done.
I have heard of health systems using algorithms as a first pass too (i.e. radiologist only see films that have had a postive in a tuned-to-be-senstive version), but that has it's own set of issues.