I guess I'm interested to see if this performs qualitatively better than GPT-3, given how many more parameters it has.

However, I think this is really a dead-end: throwing more hardware at this is just going to generate better-sounding nonsense. Yes, we are learning the "model" of the English language - which words go with which others, but successively larger transformer models don't really expose much more about the nature of intelligent conversation.

I think we need a better algorithm now.