Hacker News Reader

Comment by ZephyrBlu | Hacker News Reader

I'm extremely skeptical that people are getting the actual prompt when they're attempting to reverse engineer it.

Jasper's CEO on Twitter refuted an attempt to reverse engineer their prompt. The attempt used very similar language to most other approaches I've seen.

https://twitter.com/DaveRogenmoser/status/160143711960330240...

There's no way to verify you're getting the original prompt. It could very easily be spitting out something that sounds believable but is completely wrong.

If someone from Notion is hanging around I'd love to know how close these are.

View on HN

For the action items example, some of the prompt text is produced verbatim, some is re-ordered, some new text is invented, and a bunch is missing. Keep trying!

(I work at Notion)

Thanks for the context! That's better than I expected, but it's interesting a bunch of stuff is missing.

action items was the hardest one!!! i referred to it as the "final boss" in the piece lol

(any idea why action items is so particularly hard? it was like banging my head on a wall compared to the others. did you do some kind of hardening on it?)

¯\_(ツ)_/¯

Does it matter? Not if the prompt received represents the same embedding inside the AI. The exact wording won't matter.

The missing stuff may ultimately just be ignored by the AI.

> There's no way to verify you're getting the original prompt.

(author here) I do suggest a verification method for readers to pursue https://lspace.swyx.io/i/93381455/prompt-leaks-are-harmless . If the sources are correct, you should be able to come to exactly equal output given the same inputs for obviously low-temperature features. (some features, like "Poem", are probably high-temp on purpose)

In fact I almost did it myself before deciding I should probably just publish first and see if people even found this interesting before sinking more time into it.

The other hint of course is that the wording of the prompts i found much more closely match how I already knew (without revealing) the GPT community words their prompts in these products, including templating and goalsetting (also discussed in the article) - not present in this naive Jasper attempt.

I guess it depends what the goal of the reverse engineering is.

If it's to get a prompt that produces similar output, then this seems like a reasonable result.

If it's to get the original prompt, I don't think that similar output is sufficient to conclude you've succeeded.

This type of reverse engineering feels more like a learning tool (What do these prompts look like?) as opposed to truly reverse engineering the original prompt.

It also depends what you hope to accomplish with what you’ve reverse engineered. For example, the spectrum of acceptably usable reverse engineered gaming consoles ranges from some baseline of targets known to work, all the way to obsessive dedication to feature and bug parity. Most (not all!) emulators opt for high compatibility, rather than exhaustive. I don’t know where that high bar is for AI prompts, but I’d bet it’s more forgiving than this exacting standard. And it’s more thorough than the learning tool characterization too.

>> There's no way to verify you're getting the original prompt.

> I do suggest a verification method for readers to pursue … you should be able to come to exactly equal output given the same inputs for obviously low-temperature inputs 90ish% of the time.

This sounds like “correct, there’s no way to verify,” but with more words.

Why are you skeptical? You can try it yourself on ChatGPT: https://imgur.com/a/Y8DYURU

> There's no way to verify you're getting the original prompt.

Of course not, but the techniques seem to work reliably when tested on known prompts. I see no reason to doubt it.

thats pretty cool, its like ChatGPT is a REPL for GPT

Why would someone need the verbatim prompt?

If "prompt engineering" is going to be a thing (that has value), getting a prompt close enough to produce the same results would be what most reverse-engineers would want. In fact, not-verbatim could have advantages since you might argue you aren't infringing copyright.

https://twitter.com/DaveRogenmoser/

My god, I checked this account and almost puked. How do you become such a distilled piece of tech-bro?

I mean it's not that bad but this did stand out:

> Talking software is fun, but what we really got jazzed about was… buying all the houses around us so all of our friends could live on the same street. Anyone else doing this?

Uhm yeah loads of people are buying all the houses on a street. What planet is this guy from?

The planet of cheap money borrowing...

...are you trying to extract information from Notion's employees!? Pretty sure that qualifies as a social engineering attack! /s

i assume they're ok with it given @jitl is on this thread saying "I highly recommend using prompt injection" haha

It's the same as with generative art models that use CLIP you can do a reverse search and the prompt might not be exactly the same, but the outcome is.

If that's the goal it feels a bit pointless. If you have the skill to reverse engineer a prompt that produces similar results I assume you also have the skill to just write your own prompt.

The reverse engineering is done by the clip model and not by hand.

Oh, I thought you meant it was a similar situation to in this post where it's done by hand. Automatically generating prompts based on the output image is pretty cool.

yup there are a range of "CLIP interrogators" out there.

behold my collection https://github.com/sw-yx/ai-notes/blob/main/stub%20notes/IMA...

Oh, wow I didn't recognize it by your username at first, but I use your repository/substack a lot for research. Thanks for the work.

oh what kind of research? i am always curious how people use my “open source knowledge” repos but yeah basically people like you are why i do what i do

(i gave a prezo on the open source knowledge concept a few yrs ago https://youtu.be/XoATf7xGoUY)

Reply on news.ycombinator.com