Rather, the relationship is between what a program is and the very foundations of mathematics--formal logic. In particular, a programming language is a formal system exactly the same way a logic is a formal system. A mathematical proof is really the same sort of object as a computer program. It just turns out that the goals behind the two are different, so the rules of the formal system (either a logic or a programming language) behave very differently. But, at a high level, it's the same idea.
In fact, the way we ascribe semantics to programs and programming languages is analogous to how we can reason about proofs and logics. As a rough comparison, denotational semantics are like model theory and operational semantics are (sort of) like proof theory.
That said, for the most part, this relationship to mathematics is actually orthogonal to the sort of math you would use in writing a program. Linear algebra, geometry, group theory and so on are great for applications but not what I was talking about. Instead, the mathematics I was alluding to only comes out when you get sufficiently "meta" and start writing programs that operate on other programs--static analysis, verification and so on.
I'm not sure what you mean by "abstract tool", but programming is just as much an abstract tool as mathematics. The only difference is that we have some handy hardware for realizing the former.