Dropout seems to significantly reduce catastrophic forgetting. So does very sparse activations, supposedly. It's worth noting most NNs are trained with SGD which is an entirely online algorithm.

Not that it's a solved problem but its not unsolvable. Mostly it's that researchers don't care much about online learning since its not too difficult to train offline on stored data.