GPT-4 was created by humans, but is too complex for humans to understand. This puts it in a broader class of valuable-but-incomprehensible things that includes the world economy, the source code for large software products, and life itself.

GPT-4 was created using a rapid iterative process where vast numbers of GPU cores made tiny iterative improvements to a model of human knowledge. Indeed Matt Ridley has argued that a similar evolutionary process is responsible for the creation of pretty much everything that matters.

But this kind of productive evolution doesn't just happen automatically by itself. You can't just pour out a pile of DNA, or people, or companies and expect something good to magically evolve. For productive evolution to happen you need the right structures. Those structures ensure that iteration is fast, that each step has low risk of causing harm, and that iteration progresses in a good direction.

I wrote a blog post about how those structures work, drawing on examples from machine learning, metric driven development, online experimentation, programming tools, large scale social behavior, and online speech.