Or you're suggesting a violation of conservation of energy and momentum of light over large distances, by vacuum itself. Which would probably be detectable here. Not even sure how you can suggest it in a consistent way.
Then there's the reasonable option, that it's just gravity and the spacetime metric itself changing, which is the leading theory anyways. Explaining it away with some invisible matter that acts only gravitationally, like you suggested, is literally what they are doing.