Historically, this is accurate. The whole concept of "pure math" is actually very recent and before that math was almost exclusively tied to real world problems that often came directly from physics.
It wasn't until much later, and only in a tiny part of the world called Europe, that physics started using mathematical models. Even at that time and in that place, there were other disciplines making advanced use of mathematics and leading to exciting discoveries, starting from Economics. Jacobi who developed utility theory around the time of Newton.
Mathematics was never a subdiscipline of physics. Important parts of mathematics were developed to support models of physics, but claiming that the parts should not exist is intellectually dishonest.
Disclaimer: I'm not a mathematician.
Edit: I meant physical things like apples fam. This is an important philosophical point we don't appreciate because we are so used to them. De Morgan once wrote:
"It is not our intention to follow the earlier algebraists through their different uses of negative numbers. These creations of algebra retained their existence, in the face of the obvious deficiency of rational explanation which characterized every attempt at their theory."
I beg to differ, just today, on the road, accelerating by some negative amount made a lot of sense.