This is a great overview. Have been very keen on emulating biological neurons, where axons and dendrites may span beyond a single layer. ResNets seem like a step in that direction.