One could argue the entire field of Real Analysis was formed because Calculus showed the world that we didn't really have those definitions, but they were needed. There are cases where the integral of the derivative does not equal the derivative of the integral (violation of the fundamental theorem of calculus) without having a specific epsilon-delta definition of limit.
Also, zero is not always the same as nothing or none. Zero is an abstract number that some have decided is useful to represent nothing or an empty set, but really comes from an abstract idea that you can count nothing and have a number. This goes back to the fact that numbers are pretty useful ideas regardless if you may consider one of them a function or not.
I think this is a by-product of the way we are taught maths at the very start; that it must somehow relate to real things. We start our understanding of maths by using real world objects like apples and we show how addition works and subtraction. I think perhaps it sticks in our head that everything must somehow relate to real objects and the real world.
We somehow get past that when we are introduced to things like square roots and integrals and higher mathematical concepts, but even with those we often try and relate them back to the real world.
I wonder of there are other ways to start teaching maths that doesn't start by using balls or apples? How would that work?
[1] https://en.wikipedia.org/wiki/Banach%E2%80%93Tarski_paradox
x^2 = 1XX x^1 = 1*X x^0 = 1