Information theoretically speaking, how do you generate a "synthetic" dataset (as the article calls it) with the same fidelity as an original dataset without having access to a critical basis set of the original? What would you do to obtain that fidelity? Extrapolate from sufficiently many independent conclusions drawn from the original?

And as a followup, if you can generate a synthetic dataset by extrapolating from sufficiently many independent conclusions drawn from the original (as opposed to having access to the original itself), would you still need to use such a dataset for training?

Things like Monte Carlo simulation can be used to approximate real world conditions, but they can't typically capture the full information density of organic data. For example, generating a ton of artificial web traffic for fraud analysis or incident response only captures a few dimensions of what real world user traffic captures.

The author talks about simulating data to focus on edge cases or avoid statistical bias, but I don't see how simulated data actually achieves that.