Not really scientific or anything but I tend to give it the task: „Write a simple http server in Go that saves all requests into a SQLite database.“
What I am looking for is:
- did it forget to import the SQLite driver?
- is it doing weird SQL shenanigans like selecting MAX(id) to obtain the next potential id?
- is the code rather simple or over-engineered?
update:
Most LLMs produce a decent answer, however it you increase the difficulty a little bit by asking it "Write a simple and CGo free http server in Go ...", most LLMs get the sql driver wrong (except for gpt-4-1106-preview)