As far as I know, all adversarial attacks need to know the model... But that is a very interesting line of thought.