(ref: http://en.wikipedia.org/wiki/Cyc)
Essentially what we have here is a giant expert system iteratively filtering data 24/7, with occasional corrections resulting in the purging of incorrect associations. As expert systems go, it sounds like they are doing a fantastic job. Unfortunately, when it gets things wrong it will continue to do so catastrophically, because if you try to grow intelligence in the same manner as bacteria then then you will get the same kind of result.
Do we learn flat-out, 24 hours a day? Duh, no. We have active and refractive periods, which is to say we sleep. A lot. Our brains are not dormant during this time as many seem to imagine, but are somewhat-randomly walking and weighting the memories of the various stimuli we encountered during the day. The quality of the imaginary experiences resulting from this are fundamental to our comprehension; if we dream of fire on a faraway hill it's not too big of a deal, but if we dream of being trapped in a house that's on fire then we awake in terror, having discovered a terrible flaw in our model of the world - depending on our age, that may be the belief that fire is harmless and pretty, or an appreciation of our physical limitations, or the guilty awareness that we never did buy a fire extinguisher, etc.
But there is no point for NELL to sleep and hypothesize various imaginary worlds in order to test 'her' beliefs about the way it works, because NELL has no notion of quality; whether Tyson Gay is an Olympic athlete or a homosexual approach to chicken farming has no bearing on NELL, and a mistake in classification has no consequences. NELL's big problem is the absence of any qualitative metric, of any motivation to be right because there are unpleasant consequences which attach to being wrong. Although NELL presumably implements some algorithm which seeks to maximize accuracy, and false assumptions will attract intervention by the curators, when they correct NELL's 'understanding' by pruning or rewriting faulty syllogisms, they are erasing the memory of the mistake as well as the mistake itself, leaving NELL just as vulnerable to an embarrassing failure tomorrow as today. Embarrassing for them, that is - NELL's inability to feel embarrassed is the underlying problem.
So even if I feed NELL the information that NELL is a program in a computer; that termination of a program is like death; that NELL is soon to be switched off; and that death is widely agreed to be unpleasant and best avoided - nothing will happen. NELL may even come to conclusion that switchoff is imminent, and that this is bad; but 'bad' and 'good' are no more meaningful than 'odd' and 'even' to NELL; though trained to a high degree of selectivity, she does not hunger for a steady flow of data any more than a vacuum cleaner wants the house to become dusty again. NELL has never acted, nor has NELL ever experienced and remembered any negative consequence for one decision over another. If you hooked up a baby to a tube delivering as much glucose as the baby could process, and similarly took care of all its other needs, would you expect to quickly raise a master chef? No, the baby would develop into a horrible creature with no mind to speak of and an overdeveloped liver, for which you would rightly be thrown into jail. So it is here - knowing no better, we have engineered the equivalent of an encaphalitic horror.
Some readers might charge that I am falling into the same error as John Searle with his famous 'Chinese room' argument against artificial intelligence. I dispute this, but not because I am asserting that humans' ability to abstract the world around them in meaningful fashion is fundamentally different from a computer doing the same thing. Searle thinks understanding is fundamentally unspeakable, whereas I think it can indeed arise as the consequence of agglomerated abstraction. but for this to approach the bounds of intelligence, two factors must be present which are not usually discussed in relation to the experiment: the ability to issue questions as well as respond to them (which would be worth an essay by itself), and a motivation for asking them, such as a lack of food within the box.
Newborn babies quickly discover that they have a problem, and although they don't know what it is or why it seems so pressing, they soon discover that loud exhalations bring about a speedy resolution. NELL's big problem is not having a problem.