eoravkin

You built a cool product. I'm actually one of the founders of https://medisearch.io which is similar to what you are building. I think the long-tail problem that you describe can be solved in other ways than with live APIs and you may find other problems with using live APIs.

Out of curiosity, did you actually see any pharma ads on OpenEvidence?

Thanks a lot! We put a lot of energy into solving hallucination. The tldr is that we have a eval set to test hallucination and tweak stuff to optimize for performance on this eval set.

Thanks! Glad that you like it:)

We haven't trained or fine-tuned on neither of USMLE or MedQA. We use several LLMs in our system, and honestly I don't think there is any single thing that I can point to which is different to how other people do it. I guess it's a combination of many small improvements.

Sorry for the vague answer. We are thinking about publishing a report at some point.

We'd be interested to hear examples or ideas of how this could be useful for you!

Your point about non-urban regions is interesting. Haven't thought about that. I wonder whether it will also be much harder to show a tool like this to people there though.

Thanks! Glad you liked it. Symptoms and indications --> diangosis is a bit more challenging. When we talked to some doctors, they always said that seeing the patient directly is very important. Perhaps a system like this could be a good complementary tool, or a better solution can be built on top of it.