I also tested logic puzzles tweaked to avoid memorization. GPT3 did poorly, GPT4 got a few of them. I expect humans will still be useful until GPT6 solves all these problems.
I tested on GPT3 around Dec and Jan. GPT4 the day it came out. An example puzzle is linked below. I changed the number to 37. Instead of hairs I said it was aliens with multiple eyes. Anything to throw off memorization.