So: (Some mathematical formalism for) numbers are objects you build to act like the (informal) concept of a number.
It just so happens that the symbols that denote numbers behave like certain things in the real world when manipulated according to the rules that are customarily associated with the symbols. Those things in the real world have properties that correspond to the informal idea of numbers. But that has absolutely nothing to do with the math (except insofar as it's one of the things that makes math worth doing).
A model of a number system is an assembly of behaviors, rules, or objects which behave like the numbers you want. I don't see the circularity here, and given that they are tackling an informal concept of number, I don't see any loss of clarity either.
No, it isn't. You're just wrong about that.
Now, it is true that there are different kinds of numbers in math, but each different kind of number is formally defined. All of those definitions are interrelated, and collectively they define what it means to be a number. There is nothing informal about it.
Then you could simply point to the formalism which unites various mathematical activity under some universal model of number. I will reiterate the claim again — "number" is an informal concept among mathematicians and they are fine with that.