I'm currently setting up a small coding assessment for recruitment purposes at my company. I think in 2023 it's not reasonable to restrict the usage of tools like gpt4 or copilot. I wonder how advent of code will enforce this?
No, but it's reasonable to say "we want to get a sense of your baseline understanding of things so please don't cheat with LLMs or otherwise."
ChatGPT will, more often than not, come up with a stupidly complicated solution like to my above example that doesn't work. It takes an actual engineer to figure out why it is impossible and solve it correctly.
If I was hiring people, things like that would be great questions. It’s also so simple - keep track of the times GPT-4 has no idea of what it is doing, and use those as your questions.
In ask people to describe me the code they are going to write verbally. Or reason through an algorithm verbally. Depending on the role, I sometimes start with FizzBuzz, with a spoken answer.
They can use a notepad. They can take time to think. But they will get "why" questions and need to understand the code or constructs they suggest.
This feels safe from ChatGPT, at least for the moment.