I guess the phrase could be parsed two ways but in context it should be clear - the universe is not "infinite by definition". The definition doesn't mention size.
This is why we should all speak like Yoda. There is no ambiguity between:
By definition, infinite the universe is not.
And
Infinite by definition, the universe is not.
If the universe were "infinite by definition", it wouldn't be an open question.