> They are large, but far from overflowing.
Sure, but isn't large relative? Sure you can make them overflow in octave as well, given enough layers. Which brings us to next point :-)
> And this corresponds to a network of deep 100, which is not a realistic scenario.
Actually deep 100 is not unrealistic at all these days! https://arxiv.org/abs/1611.09326