I imagine that’s because it’s difficult for the model to decide which features are important. Understanding my human bias in looking for an intuitive explanation, the “circle” in this example looks most like the circle in the picture it identified as most similar. I imagine salience assignment is difficult without either more examples or injecting prior knowledge.