> it's like proposing to stabilize the stack of "turtles all the way down" by adding more turtles

That's completely fine. Say each layer uses the same amount of turtles, but half the size of the layer above. Even allowing for arbitrarily small turtles, the total height of the infinite stack will be just 2x the height of the first layer.

Point being, some series converge to a finite result, including some defined recursively. And in practice, we can usually cut the recursion after first couple steps, as the infinitely long remainder has negligible impact on the final result.