Our having a concept of infinity has no metaphysical significance. All our definitions are phrased negatively, by what it's not:
Infinite: That which has no limit. From the Latin 'infinitas', meaning 'unboundedness'.
Try and see if you can define infinity by what it is rather than what it isn't.
If you're saying that infinity is not directly realizable in what we perceive, then I agree. But this has little to do with how it's defined (or even its metaphysical significance); positive and negative definitions are for all intents and purposes equivalent.
Seriously, I'm not trying to proselytize nor argue. I just find atheism to be an untenable position. It's an attempt to say something while really saying nothing. Problem is, I know the same can be said of theism. But I think the latter is that much more compelling because it starts and ends with one question - "God" - that we can pose to ourselves. Where the answer leads is up to us to decide.