Curious. Apparently base64 encoding exercises were part of the training set? It would be insane if it is an emergent feature.
I asked it to generate an SVG representation of Dali's The Persistence of Memory and it output a recognizable vector image of three distorted clocks.
I happy to be proven wrong, but that certainly feels emergent.