Key word is provably. We can assume the universe is finite because we don't have evidence against it, but no one knows if it's possible to prove or reject that for good.
There is no reason to think that "infinity" will be essential to describing the universe any more than any other possible abstraction that we have not so far constructed.
Why not? An infinitely large universe goes against our intuitions, but our intuitions were formed by the experience of living on a finite planet and have no bearing on what's true on a galactic scale – just like they have no bearing on what's true at the quantum scale. There is no objective reason to assume that the universe isn't infinite.
Moreover, even if finitism is not enough, as I said in mathematics infinity is specific concepts/abstractions, and there are even infinities with different cardinalities, not to mention the axiomatic system within which it is defined produces different properties of infinity. Which infinity is the universe of all these? I am not even sure if this question can be really meaningful, unless we get to a completely different description of the universe than the ones we have where such statements can actually make sense. The best we can get imo is to show that some accurate physical model requires infinity, and that seems already quite hard thing to show.