I wonder if it's possible to prove that you can't end up with a non-deterministic system if that system is constituted only by deterministic parts? I get the sense that such a proof is possible, but I am not sure. We can trivially prove it for deterministic systems that we can model mathematically, but what about a generic proof for those systems that are beyond our modelling capabilities?

Regarding existence itself, I agree that determinism is not an exhaustive set of possibilities (if it was, it would imply an infinite chain of causes, which presents its own problems). I just challenge the idea that one can possibly get a non-deterministic system if we assume that we're working with purely deterministic components/rules.