Hacker News Reader

I assume that the savings will be passed on to students?

We're considering doing something similar in my department. Based on some back-of-a-napkin math, it won't (yet) be cheap though, at about $0.40 per query.

This is assuming the use of gpt-4 (chatgpt would be a lot cheaper, but it doesn't follow system level instructions very consistently, often spilling answers instead of replying with feedback, hints and leading questions).

Also, projected costs per query are high because we want to feed it quite a lot of input tokens. The assignment text, the student's code as well as the reference solution code.

Could you possibly reduce those costs by fine-tuning the model with the assignment text and reference code, rather than including them in the prompt?

OpenAI only offers finetuning for the pre-chatgpt models, so I doubt we'd get good enough results from that.

Something like this might work with a self-hosted (llama) model though. We do have a (small) database of student work and teacher feedback for each assignment we could use for training (privacy permitting).

So the course changes to a game of prompt injection for students aware of the state of LLMs

That's exactly what we're afraid of.

Reply on news.ycombinator.com