Hacker News Reader

Comment by wizofaus | Hacker News Reader

You're assuming that the AI is even generating anything that will make sense to a human. It seems inevitable we'll reach the point that for SaaS the AI will do everything directly based some internal model it has of what it believes the requirements are (e.g. it will be capable of acting just like a live web server), whereas for desktop and mobile apps, while that paradigm still remains relevant, it will generate the compiled package for distribution. And I imagine it would be unrealistic to attempt reverse engineering it. Fixing bugs will be done by telling the AI to refine its model.

View on HN

> It seems inevitable we'll reach the point that...

It's inevitable that we'll reach AGI. It's inevitable that humans will extinct.

Everything you described is not how today's AI works. It's not even a stretch, it's just pure sci-fi.

I'll be genuinely surprised if we don't have tools with that sort of capability within 10 years, quite possibly much sooner.

I don’t think it is inevitable we’ll reach AGI. I think that question is very much up in the air at the moment.

My point is "It's inevitable that {scifi_scenario}" always sounds kinda plausible but doesnt necessarily means anything.

Are you arguing that LLMs already provide the technology to do this or are you arguing that it "seems inevitable" to you in the sense that somebody might think it "seems inevitable" that humans will some day travel to the stars, despite doing so requiring technological capabilities significantly beyond what we have yet developed?

It doesn't strike me as being much of a leap from what we have already, certainly not compared with traveling to the stars.

> You're assuming that the AI is even generating anything that will make sense to a human.

Why wouldn't it? It's trained on code generated by humans and already generates code that is more readable than the output of many humans me included.

But why would anyone bother with using AI to generate human readable code if the AI can generate the final desired behavior directly, either on-the-fly or as executable machine code?

Because the AI's, at least right now, can't generate/change code so that it correctly does what's expected with the confidence intervals we expect. I've tried to get it to happen, and it just doesn't. As long as that's true, we'll need to somehow get the correctness to where it needs to be, and that's going to require a person.

A lot of people have already figured out at some tricks to improving code generation.

You can fairly easily update the “next token” choice with a syntax check filter. LLMs like ChatGPT provide a selection of “likely” options, not a single perfect choice. Simply filter the top-n recommendations mechanically for validity. This will improve output a lot.

Similarly, backtracking can be used to fix larger semantic errors.

Last but not least, any scenario where a test case is available can be utilised to automatically iterate the LLM over the same problem until it gets it right. For example, feed it compiler error messages until it fixes the remaining errors.

This will guarantee output that compiles, but it may still be the wrong solution.

As the LLMs get smarter they will do better. Also, they can be fine tuned for specific problems automatically because the labels are available! We can easily determine if a piece of code compiles, or if it makes a unit test pass.

Currently ChatGPT isn't, at least via public access, hooked up to a compiler or interpreter that it can use to feed the code it generates into and determine whether it executes as expected. That wouldn't even seem particularly difficult to do, and once it is, ChatGPT would literally be able to train itself how to get the desired result.

Precisely. I think people should consider the "v4" in "ChatGPT 4" as more like "0.4 alpha".

We're very much in the "early days" of experimenting with how LLMs can be effectively used. The API restrictions enforced by OpenAI are preventing entire categories of use-cases from being tested.

Expect to see fine-tuned versions of LLaMA run circles around ChatGPT once people start hooking it up like this.

> what it believes the requirements are

It will have to describe these requirements in a way that a human can understand, and verify. The language will have to be unambiguous and structured. A human will need to be able to read that language, build up a mental model, and understand it is correct, or know the way to make corrections. Who do you think that person will be? Hint: it will be a specialist that knows how to think in a structured, logical way.

Sure, I agree with that. But it will very different to how programming is done today, and I'd suggest there'll be a lower bar to becoming capable of formulating such requirements and ensuring the software works as expected than there is now.

At least permabans are going to be more fun

Reply on news.ycombinator.com