That's the point. 100 parameters is sufficient to overfit, and it's a number that's independent of the input size. Do you have a reference for your statement?
Maybe I understood that question wrong, but regardless, even if early stopping wasn't implemented, a NN would have more predictive power than the hash mapping. Both would be completely overfit on the training data set, yet the NN would most likely be able to make some okay guesses with OOD data.