I have a textbook somewhere in the house from about 2000 that says that there is no point having more than three layers in a neural network.
Compute was just too expensive to have neural networks big enough for this not to be true.
Compute was just too expensive to have neural networks big enough for this not to be true.
I'm sure that's what the text book meant, rather than any point about the expense of computing power.
Overfitting is the problem of having too little training data for your network size.