You might find the history of Doug Lennat's Eurisko informative.

"Lenat fed the massive Traveller rulebook into the system and asked it to find the best way of winning. Each night, after several hours of trial and error, Eurisko would spit out a few strategies—some interesting, some impractical, some ridiculous. At one point, Lenat remembers, it suggested he could win the game by changing the rules."

https://www.wired.com/2016/03/doug-lenat-artificial-intellig...