I'm no ML expert, but my understanding is that the bias is usually in the training data, not the model. The idea here is that you can generate bias-free "garbage" and therefore reduce the bias of the trained model

Yeah, very interesting... Like you could do a preliminary training that uses really good quality labeled data. Then almost infinitely continue training on a negative only dataset of random noise, which would theoretically hammer out any false positives that can be generated by the model.