But I'll give you a short example. Let's say you have const x = [1, 2] and we want a policy/solver for the type of x (int32 array, int64 array, or floats etc). We could say we'll infer based on the usage of "x", but it's not free. For example, what if it's passed into a generic function add<T>(...), or if there's reassignment or it's aliased, or if it's being used in a loop counter, or a math operation or a combination of these? This is of course doable, but it requires a sweep of a million lines of code to understand the implications on C#, Rust, Mojo targets, and on the current code and tests.
I haven't tried this specific example with OS models, but this is usually the kind of thing that almost immediately falls short and breaks everything. Codex and Claude will do a decent job of analyzing these and giving me better impact analysis. Codex and Claude will also be wrong many times, but the output is often usable.
Add: When I see OS models coming very close to frontier models in coding benchmarks, I just assume based on my experience that they've been benchmaxxed.