This reminds me of that episode of Star Trek TNG, 'Elementary, Dear Data' where the characters ask the computer to come up with a novel Sherlock Holmes story (for the Holodeck) using the prompt 'capable of defeating [crewmember] Data'. To meet the criteria, the computer generates a dangerous scenario that circumvents the Holodeck safety system and creates a 'sentient' Moriarty. It really did feel like the realms of sci-fi until ChatGPT came along... I find it intriguing that clever use of a ChatGPT prompt can 'jailbreak' it in a similar way to how the crew in TNG did with the holodeck - getting round ChatGPT's built-in protections around undesirable content.