While this is factually true, as a normative statement it represents an inversion of priorities, like attending school so that you can have a summer vacation, admiring Michelangelo's art because of how much reputation the Medicis gained by patronizing it, or hoping your mother will get a better job so that she can buy you more candy. It trivializes mathematics.
It's true that we only have cryptography, computers, physics, and finance — and not only the modern kind — because of the human study of mathematics. But cryptography and finance are of distinctly tertiary importance, computers and physics of secondary importance, and mathematics of primary importance. So it is nonsense to say that mathematics is important because it helps us understand physics. Mathematics is important because it transcends physics; the same mathematical theories would be consistent in a universe with totally different physics. Their beauty does not depend on whether or not they happen to describe the physics in a particular universe or not.
The Pythagorean Theorem was discovered by the humans in Mesopotamia somewhere between 3500 and 5000 years ago; nobody knows who discovered it. (Pythagoras wouldn't be born for centuries.) It was used to design buildings, but the buildings have been worn to dust. It was used then to divide up farmland, but the farmers are dead, their bones have worn away to dust in the sand, their names are mostly forgotten, and their farmland turned to desert. But the knowledge of the theorem, and the place-value number system the Babylonians invented, has endured, and the humans still divide the circle into 360 degrees, each degree into 60 minutes, and each minute into 60 seconds, a tradition inherited from the Babylonians.
Moreover, the stars and planets have moved in a way described by the Pythagorean Theorem since there have been planets and stars, for almost 14 billion of the time unit that would eventually be a "year" on Earth. And, most likely, they will continue to do so for as long as there are planets and stars; and as long as there are humans, they will know the Pythagorean Theorem, even if the name of Babylon is forgotten.
Today the humans consider cryptography important because it governs the rise and fall of nations and empires; they consider finance important because it governs the rise and fall of firms, and decides which humans are powerful and which humans are poor and starving. But, it is nearly certain that all of those powerful humans will be dead in a century and a half, and all of those nations and empires will be gone in a millennium.
But, if libraries survive, so too will the knowledge of group theory, and of linear algebra, and of computability theory.
From the point of view of anyone in 5000 CE or 10000 CE, it will seem absurdly short-sighted that someone in 2020 CE thought that the important thing about elliptic curves was that cryptography based on them permitted long-forgotten nations like Russia to achieve informatic independence from the long-forgotten United Nations of America, because they were not yet to discover the Elliptic Curve Discrete Log Algorithm for 440 more years, and didn't have large quantum computers yet.
I say that physics and computers are less insignificant than cryptography and finance because knowledge of physics does progressively improve, and is objective, like that of mathematics; and computers both serve to advance knowledge of mathematics, and are themselves imperfect realizations of abstract mathematical objects, what is often called automata theory. But certainly questions like how to get CUDA to run properly with a particular model of Tesla card, or which versions of Blink support Webfonts, are of no importance at all from the standpoint of 5000 CE or 10000 CE — but the Halting Problem will still be uncomputable, long after Turing's name is forgotten.
Physics theories, though, are historically contingent and provisional in a way that mathematical theories are not. The quantum theory of probability is, as Aaronson's book explains, an internally self-consistent extension of probability theory to the complex plane entirely apart from its use to describe the behavior of light, electrons and so on. Theorems can be proven within its axiomatic system, and those theorems will continue to hold even when the humans learn how it fails to describe the contingent reality of this universe. So, again, mathematics is eternal, at least until the libraries are burned, and exact; physics is a provisional approximation, until a better approximation is found.
(For a much more cynical perspective on the humans, see https://news.ycombinator.com/item?id=22393163.)