Comment by Yoric | Hacker News Reader

Comment by Yoric | Hacker News Reader

As someone with experience in static analysis, I find this a bit scary, for conflicting reasons.

- It's scary that it does so well. It certainly manages to explain much better than the error messages of most compilers/static analysis tools.

- It's scary that people are going to trust this. Because it is, of course, an approximation. It can misunderstand. And if you're not at least as good as the tool, you won't be able to realize that it has misunderstood.

- I expect that, by now, there is already production code relying on such tools. I would consider this a form of cutting corners. It can save you time, but you have to be really, really, really sure of which corners you are cutting. And if you use such tools, by definition, it's because you delegate the knowing to a tool that won't tell you which corners you are cutting. Which I find dangerous.

- And of course, as with all AI tools, I'm sure it's going to be used one of these days to demonstrate something laughably false in a court of law.

Using large language models (that predict text) to write code for large scale automation is how the human race accidentally destroys their large scale automatons.

So that's a good thing, right? :)

> It certainly manages to explain much better than the error messages of most compilers/static analysis tools.

It can't really compare to error messages if you don't give it code with errors.

https://whatdoesthiscodedo.com/g/6e0f858

After swapping an and b in your example it correctly detects the divide-by-zero, impressive.

After replacing the zero with an expression that must be mathematically evaluated to get zero, it still detects the divide-by-zero - that’s downright spooky.

https://whatdoesthiscodedo.com/g/6a8f359

ChatGPT’s inability to verify its claims limit its usefulness - but the example above is unsettling because it certainly seems to be evaluating things somehow, so what’s stopping OpenAI from enabling it to verify certain things through evaluation?

I'm impressed with how well it catches the divide by zero error. I obfuscated it a bit by using subtraction, bitwise and, and using randint from python, but it still catches the error:

subtraction: https://whatdoesthiscodedo.com/g/b12eef2 bitwise and: https://whatdoesthiscodedo.com/g/95a1492 randint: https://whatdoesthiscodedo.com/g/8cf88de

Are you sure it’s not just assuming and not evaluating, called your bluff and now you’re freaked out ? Sucks you can’t be sure?

I had a look at your code, it is really cool it can do that but I’m sure there are tens of thousands of examples like that in it’s training data, the example using rand was also cool but again, I can imagine it’s seen that before a fair few times too?

> It certainly manages to explain much better than the error messages of most compilers/static analysis tools.

It can't really compare to error messages if you don't give it code with errors.

Reply on news.ycombinator.com