Well see that's the other thing: "Someday" they'll have to make these connections in case they are legally obligated to erase a data source and it's derivatives.
Ephemeral is not an option then, isn't it? In face of regulation, some things can't/shouldn't be optimized. It's already the case for more established industries like pharma.
One way is to make sure the provenance is known from source tuples to all the way to the model version, and (eventually) retrain the model when any records upstream of it are deleted.
AFAIK Palatir made a business model out of this (i.e. tracking provenance for datasets and models, along with permissions).
(Just hearsay, not affiliated with Palantir in any way)