In the case that dark matter doesn't interact via any force other than gravity, then there will be an epistemological crisis in physics as your model will be indistinguishable from one in which dark matter is a particle.
Far too many people get hung up on the idea that in science what things "are" matters: because it doesn't. What matters is predictive power for future observations - if whatever you're proposing doesn't predict anything different to "invisible matter" then it's exactly as bad an explanation as "invisible matter" which so far is a perfectly good explanation for current observations.
The universe has no requirement to satisfy the day-to-day intuitions of African-plains originated upright apes, with senses optimized to spot large predators, in how it works. Quantum mechanics is a great example of that.