We found that GPT-4 performs at a very high level on dermatology specialist exam questions (which it would not have been trained on)
https://www.medrxiv.org/content/10.1101/2023.07.13.23292418v...
https://www.medrxiv.org/content/10.1101/2023.07.13.23292418v...
Which is still a significant feat. But I doubt it's doing anything more than the sort of semantic lookup that is already well-known to be within its capabilities.