These language models seem to know the words and the grammar etc, but lack a underlying concept they want to express.
There are systems that derive 'thought-vectors', but I'd be interested going the other way: somehow create such a 'thought-vector' and generate text to express that thought.
I don't know how to construct a 'thought-vector' of any concept though.