Hacker News Reader

Can’t we use the same method to generate adversarial inputs to iteratively train multiple model? After each model is generated we expand the data set by using the prior model to generate the adversarial inputs and then train a classifier maximizes the performance on both the inputs and adversarial inputs.

Now we just use n models in production and use voting for produce the label.

As n gets large, does this become robust to adversarial inputs?

View on HN

This is basically adversarial training, which is a typical (& very practical) benchmark heuristic defense for this problem. An ongoing question is to precisely characterize when and how AT works. The line of work has also proved to be very fruitful for the theoretical community & has produced very general results about problems which can be solved by neural networks, but not other techniques- e.g. kernel methods.

https://arxiv.org/abs/2001.04413

Thanks for the link. It seems like the text is focused on correcting errors across layers. I guess fundamentally there is no difference between the multi-model challenge of correcting errors across models and that of correcting errors across layers. This is dense, but I’m going to dive into the discussion around figure 14 as a starting point.

Thanks again.

Teacher student!!

:) we are all students

Reply on news.ycombinator.com