Hacker News Reader

Comment by DrAwdeOccarim | Hacker News Reader

This is a great reply because it shows just how much things are about to change. OP is likely a smart, digitally inclined individual and they missed this use case. This will sweep through the general population eventually so lean in and learn how to interface with a young AGI.

View on HN

I interface with it every day and it gives me wrong answers about 80% of the time. Is that about to change too?

I keep using it because it often has interesting clues, like the name of an algorithm I’ve never heard of, buried amongst the noise. I couldn’t imagine sending it off to do work for me without any supervision, though. Not by a long shot.

It depends. Do you ask it properly, or do you expect the proper thing from it? Using Google is a skill, just like using AIs.

Btw, GPT-4 gives me way better answers than ChatGPT. Obviously, it still needs supervision, just like Google/StackOverflow/Reddit answers. I don’t expect that supervision won’t be needed in the near future, and of course the answers still need to be adapted for the exact context.

> Do you ask it properly

Maybe not! But how would I gauge such a thing? I try to be very specific with my wording. And I’m very wary of including keywords that might have it land in a category that I don’t want it to land in. Clearly my strategy isn’t working though. Is there a resource for writing good ChatGPT prompts related to programming?

> GPT-4 gives me way better answers than ChatGPT

I might need to try that, then. I’ve only used ChatGPT so far.

> I interface with it every day and it gives me wrong answers about 80% of the time. Is that about to change too?

What kind of questions are you asking?

I've used GPT-4 since initial release, both via ChatGPT and the API, and I'm getting mostly correct answers for writing code in Rust, JavaScript and Python. It had troubles with Clojure, and sometimes the API for certain fast-moving Rust crates is wrong, but if I send the updated function signature in the next message, it can correct the mistakes.

Lately it’s been game-development questions related to physics. Mostly using JavaScript and GLSL, but sometimes Houdini’s VEX, which it probably has the worst success rate with.

I have a feeling if my domain was more mainstream I’d be getting much better results. Or maybe I just need to write longer, more detailed props or something?

In a generation or two we'll have language models that understand aesthetics and accuracy (by being trained on token sequences annotated with aesthetic/accuracy scores), and we'll be able to ask the model to generate well written, factual answers by conditioning probabilities based on high aesthetic/accuracy scores.

They did not 'miss the use case'. ChatGPT is known not to be reliable in many contexts, and system configuration/product development is not an area where you want everything to be opaque or where you should assume reliable defaults.

Reply on news.ycombinator.com