Hacker News Reader

Comment by teeray | Hacker News Reader

Every commenter is jumping in asking “is it correct?” Even if it’s not 100%, if it’s at least reasonably close, it could be a tremendous force-multiplier against obfuscation for someone with some familiarity with roughly what the code is trying to do.

View on HN

Correctness is a big deal here. This is a security context and we can assume that the obfuscators are active attackers against legible code, not just people passively hoping that their obfuscated code is obfuscated. If this becomes a popular technique, then code obfuscation tools will simply pivot to writing code that ChatGPT gets wrong when asked to unobfuscate it.

I can't even imagine that would be a particularly hard thing to do, especially if it isn't correct even before actively attacking ChatGPT! Fooling it even harder won't be terribly difficult. This is advantage attacker overall.

I imagine it would be as easy as using some cognitively loaded, but wrong, terms as variable names instead of short letters and numbers. Ask ChatGPT "please unobfuscate this network code" and get back a substring search algorithm because the network code was written with a dozen variants on "haystack" and "needle" for variable names, for instance.

ChatGPT being actively wrong would be a step back for such deobfuscators then, not a positive at all.

I don't really see this as a problem. Once you have a first cut deobfuscation from this you can refine it with other methods, like comparing input/output examples between the original and the deobfuscated version, or even use something more sophisticated like symbolic execution [1] or differential fuzzing [2] to systematically look for divergence between the behavior of the two. You could even feed these back in to ChatGPT and ask it to redo the deobfuscation given a failing test case.

Such testing won't be able to prove that the two are equivalent (unless it's exhaustive) but with decent coverage of the original you can get some good confidence. The goal of deobfuscation is usually understanding, so I'm not sure you need strong guarantees of perfect semantic equivalence with no human intervention/judgment.

And of course, existing deobfuscators have bugs and aren't guaranteed to preserve semantics either.

[1] https://en.wikipedia.org/wiki/Symbolic_execution

[2] https://en.wikipedia.org/wiki/Differential_testing

You seem to have just blipped by the deobfuscation from ChatGPT being actively wrong.

I meant what I said. I expect ChatGPT would happily output a substring search algorithm for the accept loop of an HTTP server if you just put enough "haystack" and "needle" words in the obfuscated code. How are you supposed to "refine" that into the truth?

To the extent that there is an answer, the answer is, completely ignore the ChatGPT output and use existing tools. Which is to say, ChatGPT would be worse than useless at that point.

I'm not saying ChatGPT will be slightly off, and maybe the obfuscator can kick it to be another 5 or 10% wrong. I'm saying, it is likely trivial to update the obfuscator to make ChatGPT utterly wrong, in every detail, up to and including the entire fundamental nature of the code.

When it gets it entirely wrong that will be trivially detected by an I/O example, no? So I don't see that as dangerous, just inconvenient (it sometimes doesn't work, but you know when it doesn't work). You can also use an existing semantics-preserving deobfuscator and then use that as the input to an LLM deobfuscator instead of the original.

If you're saying that obfuscators can eventually adapt, then sure. So can deobfuscators. This particular problem is kind of inherently an arms race.

A counter-counter measure could be to re-obfuscate the code with a traditional obfuscator/minifier, stripping off all the clever misleading tricks and run ChatGPT on that.

the input was generated by chatgpt. and is non-functional.

so, ROUGHLY, what is it trying to do?

I'm wondering why people are so quick to defend ChatGPT.

A lot of people are bought into the fantasy that AGI is just around the corner and don't understand that LLMs are a statistical model of words which possesses no reasoning ability.

i wish i got that much leeway in my work

Imagine a Stack Overflow where 10% of all answers look correct, but are actually completely fictional (as in, not misunderstanding the question, providing outdated information etc., but completely making up terminology and solutions) – without any cues to tell you which is which.

Reply on news.ycombinator.com