Hacker News Reader

Both RProp and Conjugate Gradient are descent methods-- they find a path to a local minima from the starting configuration. They do not help with finding the global minimum.

Using simulated annealing to guide random initializations can help find a better minima, but to get the global minima with simulated annealing takes an inordinate amount of time.

View on HN

(Poor) local minima can be sometimes reached in back propagation due to a bad setting of the learning rate and RProp does help with that. You can also repeat the learning process using one of those algorithms many times with different initial random weights and use the results of the best attempt. I think this most often suffices in practical applications of NN for getting decent results, you are right I got some of this wrong, I basically wanted to say that backprop is no longer state of the art in training and that optimization is the relatively easy part of learning.

Reply on news.ycombinator.com