This is approximately where I wound up. I had an initial knee-jerk reaction after reading the paper, but your kind of reasoning brought me back down to earth.

I don't know that I even want these things to generalize. I get it in academic terms, but from a CTO perspective how much this matters to the business? We already know our policies and procedures and aren't exactly excited about the idea of something getting curious with flipping things around.

I don't want it to generalize and start making weird assumptions like "some men are tall". I want it to say "I don't know", or provide a statistical signal indicating the same.