Consider for example writing a sorting algorithm in plain English and "executing" it on sample inputs using crowdsourcing.

If the algorithm is ambiguous, the people executing the code could use their common sense, vote on interpretations, and even get back to you if they are really stuck.

The execution would be done by hand, although some users who know how to program might write some code to help them out.

You could of course do this sort of thing at a very high-level of abstraction. Consider prototyping a word processor in this way for example.