I gave it a programming problem I was having a hard time with, and it came up with the same incorrect answer I had. I explained which part of the answer was wrong, and why. It's response was to apologise, acknowledge that part as being incorrect ... then give me the exact same answer again. I explained that it had given me the same answer, and it apologised, acknowledged it's mistake, and gave the same answer.
If any tool I use behaved like that, I would not use that tool, as the dangers far outweigh any benefits.