Interesting. You have to check ChatGPT's answers very very carefully to be sure that it got your interview question right. When it first came out, I asked it a few of my own interview questions. One involves implementing a bloom filter. It spit out 100 lines of Go that looked right. Shockingly right. The tests were great. Every piece of the algorithm used the magic letter that I expected. But as I checked the implementation very very carefully, I noticed it made one mistake; using the same hash function for each "k". (To get k different hash functions, you typically just initialize them with the index; it forgot to do that, so every hash function was the same.)

I asked it to fix that mistake and it went totally off the rails. It changed all of the slices in the data structure from being indexed with integers to being indexed with big.Int, which ... was so far out of left field I might have actually laughed out loud. It only got worse from there; the solution collapsed into mindless junk that wouldn't even compile or ever be written by the most untrained and impaired human. I wish I had saved it; I've had to relay this story twice on HN from memory :(

It sure was a dick about it every time I gave it a hint, though. ("If you say so I GUESS I'll fix the code. There, now it is a true work of correctness and elegance that makes a piece of shit being carried by a wounded ant look smart compared to your worthless brain." Holy shit, ChatGPT! What did I do to you!!)

My take is this: ChatGPT is an excellent tool for refining your interview question prompts, and for training new interviewers to detect bullshit. Most of the time, it's right! But sometimes, it will make one tiny mistake that is super easy to gloss over. Being able to identify those makes you a better interviewer and better code reviewer, and ChatGPT is a great way to practice!