"small changes in the hyperparameters can lead to large changes in training dynamics"

This is the definition of Chaos though no? Butterfly flaps its wings, hurricane on other side of the planet...