> We wanted receipts, so we invested heavily in eval infra to help us hill-climb, but also to identify when knowledge is common enough we can prune our guides.
Good god.
Good god.
We built rigorous tests to verify this actually works, and to know when models no longer need the advice.