Thanks. I guess my worry was that, once you are doing extremely well and your loss is very low, gradients are no longer independent, and will tend to go mostly up. Is this wrong?
Even then, you may have still converged on a local optimum which was the take away I got from the article