That’s why we read code. Because only there the correct and incorrect cases are clearly separated.
I.e. if by its structure we can conclude that our test cases are sufficient. Because if we don't know its structure then we don't know there aren't undisclosed sub-partitions of (combinatorially exploded) test space where it breaks.
How do we know its structure? By specifying it. How do we do that in enough detail? Code. Or we use static analysis I guess. Which is harder.
I can't even fathom how someone can say this with a straight face. Judging by the absolute garbage (architecture-wise) that the frontier models put out, I pity anyone taken in by this sort of thinking. They regularly tie themselves in knots and dig deeper holes for themselves without my intervention.
Software is exact; its maths. We would not tolerate a machine that operates on the basis proving/applying theorems right like 60% of the time. How we think this is suddenly acceptable in software engineering baffles me. CVEs can kill people, bad software can cause severe harms
But maybe I should not be on here arguing about this. I should be out there working on how to profit off such fundamental misunderstandings.