It's the same for most tasks. Where I do notice it is long agent runs, where agents take more steps and the performance difference definitely compounds over the iterations.
Most people doesn't understand how this technology works and broad regulation will mostly slow down progress. I think the safer path is the opposite. Distribute it as open source so people and companies can use it for their own cyber defense. When Hugging Face investigated the attack, the paid AI services blocked their requests so they switched to an open model on their own servers. Distribution lowers the risk not the opposite.
It's the same lesson as the Navier-Stokes credit fight earlier this month. Buckmaster and Alpoge had their unpublished drafts in private Codex sessions and OpenAI says nobody saw them but admits de-identified product data may have improved its models. There it's training data, here it's ad trackers. Either way, prompts and results that should stay private don't.
Thats why even though open models aren't perfect it has to win. You can skip the app and run the model yourself.
if the benchmarks and its capabilities are not that distant it comes to how we architect the system so that agents can do better. these newer models vary in its performance due to how we orchestrate the direction of our goal rather than how individual ability functions.