A fantasy novel with Tolkien's characters' names is an evident copyright violation regardless of how it was generated. That's not what's happening here.
If GPT is blameless doing some things because it's a deterministic model, not an agent, then the "it would be okay if a person was taught like this" defence doesn't apply in other areas