One intuition about the large GPT-models is that they are learning "facts" like they are learning grammar rules. It's just that their capacity is so immensely large as to learn facts as part of the language model.