No more than a neural network can be full. To anybody who has worked with neural networks, the following comes as no surprise:

    old information is sometimes pushed out of the brain 
    for new memories to form

Weights in a NN like the brain would just gradually shift. Older, seldom-accessed information will slowly fade.

That said, a NN can suffer from the trying-to-be-jack-of-all-trades, ending-up-an-idiot problem.