Godels incompleteness theorems can be seen as a statement about the gap between finite and infinite mathematics. Decidability, semidecidability and undecidability can be seen as the relationship between boundedly quantified arithmetic statements, statements with one unbounded existential quantifier, and statements with one unbounded universal quantifier.
Another avenue of exploring the gap between finite and infinite mathematics is via linear logic. There the thesis is that contraction, the logical reuse of variables, is where infinity creeps into logical reasoning. Indeed logic without contraction is quite tame. Logic with unlimited contraction is wild. Surprisingly there are logics with an intermediate strength of contraction: so-called light linear logics. These can classify reasoning that embodies polynomial time computation or elementary time computation. So in another sense infinity can be measured by algorithmic complexity.