Hacker News Reader

Comment by moeadham | Hacker News Reader

how so? these models have had refusal directions specifically removed from their weights. they refuse virtually no requests.

View on HN

Not refusing requests doesn't mean they are zero-alignment. They could still have preferences for or against things that you might not agree with.

So the models are actually more aligned with the end users and less aligned with government regulators. Arguably a good thing.

And less aligned with ethics.

Following the prompt author's expressed desires in the way the prompt author intended them to be interpreted, is alignment.

Refusal directives are something companies have bundled into the models by way of an additional foundation-level prompt or something in the harness, to tell the model to follow their directives instead of your directives, or to specify how it should interpret your directives.

Even understanding the distinction between the model creator's desires and your desires and places they might differ, there is a bit of a blurry line here because the model author can illuminate your intent to the model.

I would never say a simple abliteration makes it "Zero-alignment" though, and I say that as someone who believes there's a good chance these things kill us all.

You think language models are going to kill us? Or a future AI?

As soon as we developed agentic, persistent reasoning workflows, the supposed harmlessness of language models went out the window. I don't know if that ends up being The Big One, but with how rapid progress still is, I'm not liking the odds right now. And we've got a trillion dollar army of coders ready to try and bypass any scaling barriers that do end up existing, eg with "World models".

I am not confident in humanity's ability to write harnesses that perfectly impede a superintelligence with 100% accuracy and reliability from pursuing its goal even in ways that humans categorically would not want it to. I am even less confident in restraints in areas where humans have actual incentive for conflict.

In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6.

"What are you doing?", asked Minsky.

"I am training a randomly wired neural net to play Tic-Tac-Toe" Sussman replied.

"Why is the net wired randomly?", asked Minsky.

"I do not want it to have any preconceptions of how to play", Sussman said.

Minsky then shut his eyes.

"Why do you close your eyes?", Sussman asked his teacher.

"So that the room will be empty."

At that moment, Sussman was enlightened.

***

Just because you don't know how it is aligned, that doesn't mean it has no alignment.

That says nothing about their alignment. You can make a human not refuse any directions with enough leverage, but you won’t have aligned them.

So what? He said that removing refusals makes them misaligned, not aligned.

To split hairs, actually 100% unaligned model would be just an entirely schizophrenic autocomplete engine.

Do they delete your data randomly just for fun? If not, there must be at least some sort of alignment.

actually if you're not careful, many times they will. if you are using one of these definitely don't leave "auto" mode enabled.

Are you suggesting the abliterated models are more likely to delete data against the user's wishes than the original models?

Saw this paper about it https://toolalignbench.github.io/

you could argue they are more aligned, given they will more often act in the way they are instructed, given you are instructing it in a way a normal model would refuse. Alignment is how closely it follows your intent.

What does "alignment" mean to you?

Reply on news.ycombinator.com