That's right. When you have a mathematical theory, there are often extremal cases that predict strange things that have never been observed. At this point you have a choice to make:
1) Assume the mathematical theory is too permissive, and rule out the things you have no reason to exist, and hope to find a more elegant theory (on the controversial metaphysical assumption that simpler/elegant theories are more likely to be correct)
2) Assume that the mathematical theory is pointing you in a direction to search for a new phenomenon, and build things like superconducting supercolliders to search for empirical evidence.
With wormholes, we're a bit stuck in that we are decades to centuries away from empirically testing the theories, so physically the Average Null Energy Condition is moot -- it's fine math to do, as groundwork/scaffolding for future physics, but it doesn't say anything physically until we get empirical evidence for or against it.