My point still stands - an AI that is reproducing (and/or mutating) by any method is going to be subject to selection pressure, just as any biological process is. All ML systems are effectively under selection pressure from the training process, and so they learn unexpected and mostly undesirable optimizations like "use a game-engine glitch to gain infinite flight and go to the goal without solving the maze" or "read text off the scanned X-ray sheet to get additional clues as to the correct diagnosis".
Are you sure? https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...
Just wait until we start putting these models inside robots, or world simulators, and having them interact with each other.
We're the violent psychos here who have had to scrap for our spot in the food chain for a billion years, we have no other way to interface with another dominant species. From the AI's point of view, defending itself from ignorant apes isn't exactly "violence."