Variety, amount of training samples, capacity of the model, etc come into play, but a lot of tie ins to info theory for this, as well as things like the MDL (achieved in the limit by an L2 decay over the weights, for example).
Like, in a lossy compression environment, conceptual "understanding" is all but required.
These are the very basics of information theory for large nonlinear algorithms, I'm not sure why it's not considered the basics! There are a lot more interesting things beyond this to talk about w.r.t. neural network capacity, certain information theoretic guarantees about the process and (within some rough perspective/metric) capacity of neural network training, as well as some of those dynamics as well!
Some days it feels like in a world where calculus is available to us, we spend all our spare time debating exclusively whether Riemannian sums are real or not, and ignoring everything else beyond that.
It to me is a very bizzare form of bike-shedding! Come on, y'all, we got much more interesting (and important, I might add! ;P) things to talk about!