This is a recreation of a fictional computer program from the excellent Apple TV show - Severance.
The work is mysterious, and important.
Season 2 is going now. It’s one of my top 3 shows of the last decade, highly recommend it.
12,232 karma · joined October 5, 2008
gmail » teej.murphy@
current » Linear
prior » Downlink (YC W24), Chaturbate, Minted, Zynga, MinoMonsters
The work is mysterious, and important.
Season 2 is going now. It’s one of my top 3 shows of the last decade, highly recommend it.
But this approach to teach chain-of-thought doesn’t do that. In this post, they take a small model (7B) that already knows math. Then they give it a relatively small number of problems to solve (8k). They use a simple reinforcement learning loop where the only goal is to get the problem right. They don’t care how the model got the right answer, just that it’s correct.
This is part of the “recipe” that DeepSeek used to create R1.
After many iterations, just like DeepSeek, they found that the model has an “aha” moment. It starts emitting chains-of-thought where it wasn’t before. And then it starts getting the math answers right.
This is the gist of it. I don’t fully understand the recipe involved.
Can you teach small models to “think” just using RL? How small can they be? What tasks does this work for? Is just RL best for this? RL+SFT? Everyone’s trying to figure it out.
We have observed that LLMs can perform better on hard tasks like math if we teach it to “think about” the problem first. The technique is called “chain-of-thought”. The language model is taught to emit a series of sentences that break a problem down before answering it. OpenAI’s o1 works this way, and performs well on benchmarks because of it.
To train a model to do this, you need to show it many examples of correct chains of thought. These are expensive to produce and it’s expensive to train models on them.
DeepSeek discovered something surprising. It turns out, you don’t need to explicitly train a model to produce a chain of thought. Instead, under the right conditions, models will learn this behavior emergently. They found a way for a language model to learn chain of thought very cheaply, and then released that model as open source.
Thought chains turn out to be extremely useful. And now that they’re cheap and easy to produce, we are learning all the different ways they can be put to use.
Some of the open questions right now are:
- Can we teach small models to learn chain-of-thought? (yes) How cheaply? On which tasks?
- Can we generate thought chains and just copy/paste them into the prompts of other models? (yes) Which domains does this work for? How well does it generalize?
That’s what this post is going after.
Trends come and go. The stories and comments come and go. HN is constantly changing. You can’t attach yourself to how something was before. It will change again.
Thats just life.
This is a problem when people naively use "give an answer on a scale of 1-10" in their prompts. LLMs are biased towards particular numbers (like humans!) and cannot linearly map an answer to a scale.
It's extremely concerning when teams do this in a context like medicine. Asking an LLM "how severe is this condition" on a numeric scale is fraudulent and dangerous.
Metformin is one of the most widely studied drugs on the market. I can't find any study that links it to PSP (Progressive Supranuclear Palsy).
Has anyone made a political censorship eval yet?
"Following an investigation, we have now confirmed that certain information relating to customers of Santander Chile, Spain and Uruguay, as well as all current and some former Santander employees of the group had been accessed. Customer data in all other Santander markets and businesses are not affected."
By default, no. But it is standard operating procedure for sales engineers to request and be given access to customer data so they can build demos.
I wrote a better version here: https://gist.github.com/titan-teej/924dcb42604a98b90d6419262...
TL;DR they don't check for users who can transitively gain admin access.
TITLE TOTAL HRS WATCHED
----- -----------------
Ginny & Georgia 967,200,000
The Night Agent 812,100,000
You 766,300,000
Outer Banks 740,400,000
The Walking Dead 738,600,000
The Glory 622,800,000
La Reina del Sur 616,800,000
CoComelon 601,200,000
Suits (2011) 599,100,000
The Blacklist 596,900,000
Manifest 581,900,000
Grey's Anatomy 560,300,000
Wednesday 507,700,000
Gilmore Girls 505,800,000
Breaking Bad 505,000,000
Queen Charlotte 503,000,000
Friends (1994) 448,500,000
Love Is Blind 439,300,000
Lucifer 434,300,000
The Big Bang Theory 420,400,000
Shameless (U.S.) 392,600,000
PAW Patrol 392,300,000
New Amsterdam (2018) 375,500,000
Brooklyn Nine-Nine 358,900,000
Firefly Lane 342,700,000