What does the chatbot say, when asked to do a complete course assignment question?
Just kidding, of course.
In all seriousness, when I asked GPT-4 to solve a problem from the 2020 version of CS50, not only did it spit out a correct solution, it also correctly used "#include <cs50.h>"
I imagine they'd put some kind of guardrails up against literally spitting out the complete solution. I kinda doubt CS50 students would be sophisticated enough to a) realize that they're talking to GPT-4, and b) try to jailbreak it to give them the complete code. However, if they did, and they said they did, as a teacher, I'd be inclined to give full marks for the solution, provided they also tested it and it worked. ;)
To my knowledge, nobody has any good answers about using LLM to cheat, but I would not use that as a reason to avoid them for teaching.
The cheaters aren't going to ask ChatGPT-CS-50 for the answers, they're going straight to GPT-4.
i think that's one of the real danger of this stuff: society used to put some real value into your intellectual efforts - that was the stuff that promised "the Future", that's a real return on investment. Now these large language models started a kind of inflation in this area of endeavour. I would guess that the younger ones will have to ask themselves the following question: "why bother, if the LLMs will catch up on all of your efforts within five years, or so?"
Earlier generations didn't have to ask themselves this question.
I am trying to teach my kids some stuff, but this question is always lurking somewhere in my mind: I think this effect of a general demotivation is the real danger to our civilization - not these unrealistic notions of a 'robot takeover'.