If you speak math problems to your child you are… wait for it… teaching them arithmetic. That’s what teaching is.
Arithmetic is not something that just naturally exists in the world, it’s a purely human construct. It does have to be taught in one way or another, otherwise people won’t know it. And, unlike language, I seriously doubt parents are just speaking arithmetic randomly in conversation that often. Your kid most definitely will not just “pick up” arithmetic.
In fact, I doubt a kid would even pick up language if you never spoke to them directly, and instead they just heard bits in passing between adults.