Indeed, another weakness is that multi-layer perceptrons and their deep counterparts are susceptible to "catastrophic forgetting" -- training away old memories when new training data is presented. This is one reason why it's necessary to repeatedly run through the training data. This is a result of the way learning works in these neural networks. Like you say, humans have the ability to learn associatively in one-shot, a very different learning method.

There are neural networks like Grossberg's Adaptive Resonance Theory (ART) which do not suffer from this flaw.