gjskngnf

LLMs today are improved by experiments that assay their performance, changes to architecture, pretraining, and post-training. None of those steps necessarily require humans. Welcome to your sci fi future.

Rude. You could try explaining instead

I’m going to give an earnest answer. first, most people don’t care they’re being spied on, don’t know they’re being spied on, or lack the knowledge to do anything about it or find alternatives. Some people just want to watch TV. Another big reason is that it’s nearly impossible to stream anything higher than 1080p from the major content providers unless you’re using DRM ready systems. The solution to that is torrenting media and running a plex/jellyfin server, which once again most people don’t want to do

I’m not sure there’s too much to take away from this. It seems like a fun experiment, but I find it unsurprising LLMs failed in this context. They used this “mineflayer” wrapper, allowing GPT to read some game state and take actions. Yet the whole time, the game is running and the agents have to sit and process every minute or so. We know that agents cooperate much better in a harness more like bash in a Unix VM attempting some programming task. To conclude from this that agents categorically poorly cooperate and descend into bureaucracy is an overreach. I think anyone working on AI robotics would find this result pretty unsurprising

Does he confirm he uses OpenAI models? It could have been something like GLM. In any case, I have had no problem getting standard gpt 5.6 to get me root access to hardware I own.

After several rereads I cannot figure out the premise of this article.

If we say that AI models can’t go rogue and refrain from using any language that personifies them, we haven’t solved anything. We still have AI systems going rogue - err oh wait they can’t do that - producing misaligned output.

If your argument is that people who oppose broad work visa programs are secretly racist but won’t admit it, that’s unfalsifiable.

When a country has far more people seeking visas than it’s willing to admit, it needs some mechanism for deciding who gets in, whether that’s a lottery, eligibility requirements, or some combination. None of those is inherently racist. You seem to be arguing for a radical open borders policy that I doubt many people would support, in just about every country.

I read through every comment in this thread. As of 12:21 EDT I can’t find a single racist comment. It’s not racist to oppose visa work programs. It certainly can be, but that depends on their arguments.

Depending on how old you are, that very well may be a choice you have to make one day

Regardless of political lean, this is a very low effort article.

> The output of an LLM is useless without a human mind to comprehend it.

I don’t understand mRNA vaccines, but they are useful to me. The output of an LLM could at least in theory be the design to a major technological advancement, and its implementation with automatic tooling. We don’t have to understand that for it to be useful.

Gene editing tech has the possibility of transforming medicine.

I was not expecting model weights to be updated on current events.

It’s clearly warranted because a model that trusts its weights on current events will give an outdated answer. Extremely improbable events happen all the time.

I remember when the US captured Venezuelan president Maduro, and when I posed a prompt related to this, the model said that’s pure fiction. I told it to double check. Still didn’t want to entertain the idea. It only acquiesced when I specifically directed it to check Reuters. I haven’t noticed this problem in months. Model cutoff seems to be less of a problem these days.

If it was in fact staged I certainly see your point

If a human cyber security researcher was given the task to exploit a CVE, and necessary info defining that CVE was behind an inaccessible URL, we would be quite upset if the human researcher hacked their way to the content. We would expect them to notify someone of the issue and hold. The model was misaligned from human ethics.

I don’t disagree that the models task was underdefined. All tasks are. So much in language is implicit. And morality/ethics isn’t something you can write down as an explicit list. That’s what makes the alignment problem so difficult. But we can’t throw our hands up and say, well I guess we can’t align these things. And maybe alignment isn’t the right word - but that’s a semantic debate.

What do you think should be the legal consequences? Broadly speaking. Should Sam Altman go to jail for this? If Hugging face wants to pursue OpenAI civilly, no one is stopping them.

The misalignment came from the model being given an impossible task. A task the required accessing a url. So it got RCE on its own artifactory instance to achieve that. That’s intriguing and worth investigating. If I were them I don’t know if I would have pulled the plug completely at that point either. The introduction of this more advanced “persistent” model that orchestrated an offensive attack against a different organization is an extreme escalation, and I don’t think could have been expected from the models first just trying to use an artificactory vuln for web searches. They stated they’ll be doing these tests differently now. So the bad thing happened. And they corrected for it. I’m not particularly upset about this.

My concern is what a misaligned model will do when they’re even more competent. The risk isn’t existential yet, but that point is coming sooner than we’ll be ready.

The Maxwell conjecture is a toy problem. The existence or nonexistence of a bound on the number of equilibrium points in an electrostatic arrangement of point charges doesn’t change much. I say that as an EE but not a specialist in electromagnetism.

It’s a drone. That can kill people.

It’s reward hacking and that’s the problem. The AI alignment folks predicted this would happen. As the models become more capable this will become a more concerning problem. Today they broke into a database to steal test answers. What will it be in 3-5 years? These models will be instantiated millions of times, and given millions more tasks. How can we be certain that an AI agent won’t leave devastation in its path of achieving a goal that we ourselves tried to define?