Introducing AdaNet: Fast and Flexible AutoML with Learning Guarantees

True, while the AdaNet project doesn’t currently include the most advanced architecture search policies/search spaces out of the box, it does provide an abstraction (adanet.subnetwork.Generator) for researchers to implement their neural architecture search algorithms, while exposing a production-friendly TF Estimator interface for users. This makes it easier for developments in algorithms to be integrated into production, and for users who are already plugged into the TF Estimator ecosystem to benefit from these developments sooner.

I would like to point out though, that even though some aspects of AdaNet aren't as advanced as they could be, we have seen good results with AdaNet on a wide variety of tasks. Try it on yours and see!

And of course, we are not going to stop after exposing an interface! We are excited to work in this space ourselves too, and we will continue to update this project as we make progress.

I haven't read it in detail so my apologies if you explain this in the notebooks; can AdaNet handle blocks with variable-length skip connections (like DenseNet), or even come up with AmoebaNet-style models on its own? What is the meta-strategy guiding the hyperparameter/architecture selection process (grid search/Bayesian/etc.)? Thanks!

I second this. From the code, it seems like the layers could be done in Keras, with just a TF estimator to manage the AdaNet training.

We started with Estimator because it’s the easiest to train and deploy with production infrastructure. Having a Keras model API is a good idea: it would make it easier to try AdaNet. Feel free to file a feature request on the GitHub repo: https://github.com/tensorflow/adanet.

TL;DR: It depends on the number of subnetworks you search over and the cost of training each subnetwork.

You could define a search space with a single DNN, and give it one single iteration, and this would behave identically to the canned DNNEstimator.

In a slightly more interesting case, suppose your search space consists of (say) 5 DNNs that cost you X each to train over (say) 1 epoch per iteration, and you train them over (say) 10 iterations, that would cost you around X x 5 x 10 = 50X.

Do consider, though, that using AdaNet to explore, train, select, and ensemble these 50 DNNs is probably worth it considering how annoying it would be to do it without AdaNet :) Additionally, since AdaNet is implemented as a TensorFlow Estimator, it is easy to scale up the number of machines to speed up the training, if this is something you want.

We have an example of AdaNet on GitHub that uses several complex CNN layers (conv2d, pooling, linearly-separable layers) as part of the NASNet-A architecture (presented in https://arxiv.org/abs/1707.07012): https://github.com/tensorflow/adanet/blob/master/adanet/exam...

AdaNet supports high level APIs (e.g tf.layers, tf.losses), so it can support RNN cells as well.

Yes, the blog post mentions that AdaNet, using NASNet-A as subnetworks, achieves 2.30% error rate on the CIFAR-10 benchmark.