Agreed. For starters population growth is part of the equation.
Consider the following scenario. There are 10 people in the world, and $100. So all being equal [1] each has $10 to spend. (Remember "money" is just a useful way of measuring accumulated social value.)
Fast forward a few years, now there are 100 people, all creating social value. To maintain the status quo the money supply must now be $1000. 0% inflation as the money supply per person is still $10.
[1] now imagine that all things are not equal. Some people add more social value than they consume. Ie their accumulated $ grows. In effect some number of $ are no longer in circulation. So more money has to be added to the supply to compensate.
Equally some company or individual may move money out of the economy and store it externally. Think cash under a matrass, or sitting in a Bahamas bank account. That money can (and must) be replaced in the system or the system will fail.
So the increase of money supply, taken as a measure by itself, is a meaningless number.