My first thought for getting around this was something like formal verification that a black box does what it says—but once 'what it says' is too complex for a person to really understand (e.g. a black box containing the behavior of a human brain), you've hit a wall with this approach.
The other thing I can imagine is building up 'bicycles' which have parts of proven-correct medium complexity, arranged together in a single higher-level system also of proven-correct medium complexity (where medium complexity is: more complex than bike parts, but still with readily human-comprehensible semantics). I'd characterize these systems as sort of like Iron Man suits. Maybe that doesn't really add anything of practical benefit beyond the original bicycle analogy though ;)