I guess the phrase could be parsed two ways but in context it should be clear - the universe is not "infinite by definition". The definition doesn't mention size.
By definition, infinite the universe is not.
And
Infinite by definition, the universe is not.