Hacker News Reader

Hacker News

Comment by userbinator | Hacker News Reader

Full thread

userbinator

The implications of this sentiment are disturbing.

It is considered an "attack" to probe at something to understand how it works in detail.

In other words, how basically all natural science is done.

What the fuck has this world turned into?

View on HN

We used to call this "reverse engineering". I'm not familiar with what the law says on reverse engineering, but calling it "stealing" seems a bit too much.

"Reverse engineering part of a production language model" has a more boring ring to it though.

Attack here is a term specific to cryptography and infosec, it does not imply doing anything illegal or violent. "Oracle attack" is the more specific term where you probe a system to give up some of its internals it was not meant to expose.

Wording matters. To use LLM terminology, "attack" here may mean something neutral, but it's helluva closer to "fear" and "malice" and "evil" in the latent space than terms like "probing", "studying", "examining", or "reverse engineering".

FWIW, cryptography and infosec as fields get a good mileage out of exploiting fear.

That is only true in everyday latent space/vocabulary. In the ones underlying arxiv papers, cryptography and LLM, attack has the latter meaning.

Perhaps, but everyday vocabulary is what public policy and law discussions happen in, and I think that's what 'userbinator is worried about (and so am I).

See also: "piracy is stealing" or the everyday vocabulary meaning of the word "hacker".

It's getting information the org that created and hosts the model doesn't want you to have. Just because you think that information should be shared doesn't make it any less of an attack.

OpenAI has blatantly said that the "open" in their name was a deceptive marketing ploy. At this point in time, they aren't interested in sharing much if any real research, and trying to discover this information is now an "attack" against them.

OpenAI didn't write this paper.

One of the co-authors is from OpenAI.

> "open" in their name was a deceptive marketing ploy

It is now, but it hasn't been since the beginning. Hence the elon lawsuit.

The United States.

Do you consider blind SQL injection an attack? If a user is not meant to have access to information, but they can somehow access it, that is a vulnerability.

Sure, except the information here is not other users' account info or something, it's analogous to the source of the DBMS. Which is not typically considered sensitive.

Tell that to Oracle…

No, it's like the schema of the database which is normally a secret.

Reply on news.ycombinator.com