Regularizing neural networks with dropout and with DropConnect

Neural networks are non-linear function approximators that use artificial neurons with connections between the neurons to do calculations. Dropout is a technique in which one randomly turns off neurons during training to make the network more generalized by making neurons less strongly interdependent. DropConnect furthers this idea by, rather than turning off entire neurons, merely turning off single connections between neurons.

That being said, I follow Hinton pretty closely, I am a bit surprised he or one of his students didn't think of this generalization when developing dropout in the first place! It seems like a pretty natural extension.