You should require positive proof that code does what you want.
Similar to how eg Haskell's type systems doesn't have to solve the halting problem [0]: it just rejects some programs that would be ok, but don't conform to the type system.
See also proof-carrying code. https://en.wikipedia.org/wiki/Proof-carrying_code
[0] Ok, unless you use undecidable instances or something like that.