Also isn't growth a natural result of human progress? As science and technology move forwards they allow for more productivity per person and thus drive growth right? Or is that an overly simplistic notion?
Per capita growth is relatively recent in the long arc of history. (Last 300 years?)
Yes, but the whole issue is how we are running out of mineral/natural/ecological resources. And at an exponential speed.
Progress also means that we can use resources more efficiently, e.g. by recycling them. By limiting progress (and growth) you're also making resource use less eficient, so that we'll run out of them faster.
Jevon's Paradox means that increasing efficiency also increases resource use, and us running out of them faster !
Environmental damage is a problem but we certainly aren't running out of most minerals. Mineral extraction volume worldwide is steady or increasing for most types.
You're right, I should have been more careful using that term - but for how long ? Last time I checked, we only had a few decades left of potential resource extraction growth before we hit ~half of eventually extracted resources, and the production would start to decline.