This is a hard question to answer because mathematics is a broad subject. The fact that you think libraries will exist or be important but that you don’t mention eg bignum support suggests to me that you want to do stuff with floats and matrices and differential equations. Julia would be an excellent choice for this. Language stability is something they care about because they want the language to be suitable for reproducible calculations in scientific research.

If you’re more interested in pure mathematics—things like (non-analytic) number theory, algebra, combinatorics—I would recommend Haskell. I find the type system helps a lot for algebra and the freedom to express computations in more natural and functional ways helps with things like number theory.

Both languages are fast enough. The thing you should mostly care about is the speed of writing correct programs rather than execution on large datasets anyway. Julia is in many ways like python except you don’t need numpy because the built in arrays and for loops (and auto broadcasting) are fast enough.

If you want a proof assistant then I don’t know enough about the available options to make a good suggestion