I agree that numbers don't exist in the real world. However, we humans have an ability to perceive the world through numbers. The question then is about our perception of the world if you wish.
In other words, it is an issue between access and existence. Does access need existence? For instance, when X says 'John is charismatic', is 'charisma' like 'neurosis'? Definitely not, and X sees John's charisma. And this charisma doesn't play the causal role.