It's the classic failing of everyone's casual "alternative to dark matter". It's not X because we can't detect it, what if instead it's an undetectable form of Y?

Far too many people get hung up on the idea that in science what things "are" matters: because it doesn't. What matters is predictive power for future observations - if whatever you're proposing doesn't predict anything different to "invisible matter" then it's exactly as bad an explanation as "invisible matter" which so far is a perfectly good explanation for current observations.