The answer to the paper's question is likely yes—especially if context is used effectively and memory and summaries are incorporated. In that case, chatbots can complete even more complex games, such as Pokémon role-playing games [0].
The challenge with benchmarking text adventures lies in their trial-and-error nature. It’s easy to get stuck for hundreds of moves on a minor detail before eventually giving up and trying a different approach.