NickAndresen

"They have threatened to remove us from their systems if we maintain these safeguards; they have also threatened to designate us a “supply chain risk”—a label reserved for US adversaries, never before applied to an American company—and to invoke the Defense Production Act to force the safeguards’ removal. These latter two threats are inherently contradictory: one labels us a security risk; the other labels Claude as essential to national security." from Dario's statement (https://www.anthropic.com/news/statement-department-of-war)

On Thinkish, Neuralese, and the End of Readable Reasoning

When OpenAI's GPT-o3 decided to lie about scientific data, this is what its internal monologue looked like: "disclaim disclaim synergy customizing illusions... overshadow overshadow intangible."

This essay explores how we got cosmically lucky that AI reasoning happens to be readable at all (Chain-of-Thought emerged almost by accident from a 4chan prompting trick) and why that readability is now under threat from multiple directions.

Using the thousand-year drift from Old English to modern English as a lens, I look at why AI "thinking" may be evolving away from human comprehension, what researchers are trying to do about it, and how long we might have before the window gets bricked closed.

Subtitle: On Thinkish, Neuralese, and the End of Readable Reasoning

When OpenAI's GPT-o3 decided to lie about scientific data, this is what its internal monologue looked like: "disclaim disclaim synergy customizing illusions... overshadow overshadow intangible."

This essay explores how we got cosmically lucky that AI reasoning happens to be readable at all (Chain-of-Thought emerged almost by accident from a 4chan prompting trick) and why that readability is now under threat from multiple directions.

Using the thousand-year drift from Old English to modern English as a lens, I look at why AI "thinking" may be evolving away from human comprehension, what researchers are trying to do about it, and how long we might have before the window gets bricked closed.