The text generated by GPT-2 is far superior to HMMs. GPT-2 was able to perform unsupervised machine translation and answered more than 5x as many questions correct on the SQUAD Q&A dataset than the previous best pure neural model.

Not to mention that the text generated by GPT-2 can often fool an online reader whereas HMMs have the problem of being long-term incoherent and don’t reference back to subjects of the sentence like GPT-2 often does.

I’m not staying you should believe the AI hype in news media. But the paper does contain a lot of thorough analysis and comparison to the previous state of the art.

https://cdn.openai.com/better-language-models/language_model...