"Thought" would indicate it knows what the words mean - I highly doubt it, even with the massive size of the network. It just knows which word is coming next, given the previous one.
Your brain also just "creates the illusion" of understanding. It's slightly better it to the outside world, and a lot better to itself. But it's really just a matter of degree.