Also check out this thread: https://x.com/tszzl/status/2109822082024656927

The current pace of advancement and the fact that fresh writing data will likely not be a requirement at all to continue accelerating. The pretraining paradigm no longer needs this.