In ML, this will bump you into a nearby optima at random. That optima may or may not be better than your prior. And if it’s worse, there’s a good chance it’ll be very difficult to get back to where you were without external checkpoints to revert to.

The brain is the same, but no checkpoints.