> (1) a different measure for results, or (2) give an objective mechanism for evaluating the "worth" of a technique that doesn't involve looking at results.

One does science also to understand why something works (i.e. what is the theory behind it). Deep learning gives something that (perhaps) works, but doesn't give an any clue to the question why it works - on what theory does the neural net internally compute in its black box.

Based on this criterion one can even argue that deep learning papers should not be considered as science as long as the authors of the deep learning paper make no really hard attempt to present at least a partly understanding of the internal theory that the trained neuronal network executes (i.e. a very detailed interpretation of the weights with ideally falsifiable predictions). This is hard work, I know, but isn't that nearly anything in science?