Not knowing that there is an unlimited number of numbers becomes a problem. Trying to do any math without allowing that between any two numbers there's another number is pretty difficult.
There are people who derive a mathematics without the axiom of infinity, but it's weird, distorted, unnatural and doesn't turn out to be as useful as often as the regular variety of math.
Feel free not to believe. Just don't expect to do much advanced math.
Oh, and if you use proof techniques in programming, such as loop invariants or transformation theory, you are using infinity, you just didn't realise it.