If I read thousands of books that explain the details of another civilization in another galaxy, very thoroughly and consistently, but it it just happens to be all made up - did I gain knowledge? More importantly, does what I have in my brain now flip from being fiction to being knowledge if that civilization flipped from not existing to existing? How so, if nothing in my brain, or how I live out the rest of my life, changes in the least, if not a single atom in this galaxy changes (let's ignore that gravity has infinite reach and all that, for the sake of argument)?
If yes, how? What in your definition of knowledge makes that possible?
sounds a lot like math - made up entities that very thoroughly and consistently fit together.
Edit: for "not in the training data" yes, humans generally can't know what they can't know.
Its necessary if you want to replace such a worker, at least to have the corporate culture and knowledge in its training set.
Internet started it, hopefully LLMs will finish it.
Now with LLMs lowering the average through cognitive offloading and skill atrophy, prepare for it to get a whole lot worse.