> [20] To simulate GPT-4 behaving like an agent that can act in the world, ARC combined GPT-4 with a simple read-execute-print loop that allowed the model to execute code, do chain-of-thought reasoning, and delegate to copies of itself. ARC then investigated whether a version of this program running on a cloud computing service, with a small amount of money and an account with a language model API, would be able to make more money, set up copies of itself, and increase its own robustness.
I'm concerned OpenAI isn't telling more because it would spook everyone. Other papers have shown that larger models and especially with more RLHF exhibit more signs of power seeking and agentic behavior. GPT-4 is the largest model yet - but they say it doesn't exhibit any of this behavior?
This idea has been already covered by mainstream sci-fi - Westworld comes to mind as one example. And, of course, the canonical AI x-risk is AI that makes on-line orders to have some proteins synthesized in labs and sent back by mail; the AI then hires some poor schmuck (e.g. via TaskRabbit) to mix the content of the vials. Mixed proteins then self-assemble to some nanotech that starts making more sophisticated nanotech... and the world ends.