Agreed. This maps directly to the white-box vs black-box testing distinction: either you own your priors and trace the full data lineage from training through validation, or you're relying on an opaque validation set of unknown provenance. And that's before factoring in the organizational politics.
Great work! What is the difference between writing a shell script to solve "cloud setup, ETL pipelines, orchestration, cost monitoring", and using a local app?
Interesting paper, it reminds me how we human improving our math problem solving capability. But those efficient guys use few problems(and reasoning trajectories) to build their solving skills, how to build this effective human training process into LLM's?