This reminds me of dropout regularization - the key idea there is to randomly turn off units during training.