Hacker News Reader

Yea this is what I was thinking of. Another work in that direction: https://www.nature.com/articles/s42256-021-00338-7

I never heard of the tanks story, but I vaguely remember stuff in radiology which was a similar story basically, it learned some spurious correlation to the label.

"This study suggests that the use of surgical skin markers should be avoided in dermoscopic images intended for analysis by a convolutional neural network."

Or you train with images where those markers are already removed by another NN? Or use some photos without cancer but with markers in the mix.

I heard a similar story about AI being used on MRI scans.

What the researchers hadn't realised was that the samples were coming from two different models of MRI machine and that patients with more serious symptoms / worse expected outcomes were more often being sent to be scanned on the more advanced of the two.

The AI had just detected some hidden difference between the outputs and was infering that the patients scanned on the expensive machine were more likely to have a serious condition.