after having played AI dungeon for a while these issues become obvious very quickly. These large models are good at producing text in some probabilistic manner but they don't understand anything at all.

Other than for obvious commercial and hype reasons I have no idea why there is still so relatively little recognition of criticisms that Gary Marcus and others provide because it's blatantly obvious if you interact with these systems even for a few minutes that they're to intelligence what a trojan horse is to a real one.

There's no real understanding of the world, no coherence, no real ability to grasp logic or have some ontology of the world and that makes reasoning about the world other than producing some 'nice sounding BS' as the article puts it impossible.