This statement: "Or in other words: the model, its size, hyperparameters, and the optimiser cannot explain the generalisation performance of state-of-the-art neural networks." is not true and very misleading. Careful selection of hyperparameters and the model can clearly improve generalization - the article is making a mistake in assuming that getting to zero training error is a good thing or a desirable thing. In fact a large part of hyperparameter optimization are choices that ensure generalization, and some of the fundamental choices such as early stopping and many others do determine how well the model generalizes. If your model has zero training error you have likely made poor choices.