your point is valid but the paper explains it clearly and obviously. they are NOT dimensionally reduced hyperparameters, no. The hyperparameters are learning rates, that's it. X axis, learning rate for input (1 hidden layer). Y axis, learning rate for output layer.
So what this is saying, for certain ill-chosen learning weights, model convergence is for lack of a better word, chaotic and unstable.