I had conversations in Morse code with like gpt 3 or 4. I guarantee there were no snippets of what we were chatting about, in morse code, in its training set.

LLMs are clearly able to apply seemingly arbitrary transformations to symbol streams in ways that generalize well beyond the training set. It's getting into metaphysics whether that is "novel" but it's definitely not "copy paste from the training set"