Stopped reading at "manual tuning is going the way of dodo, and good riddance". The guy clearly has no idea whatsoever what he's talking about (figures: he was an SRE at Google, nowhere close to any ML/DL efforts).

Hyperparameter tuning is an area of active research and a dark art of sorts, and it will remain that for the foreseeable future for one simple reason: hyperparameters are interdependent and _data dependent_ as well, and nobody, not even Google with its TPUs has the compute to tune them "automatically" to any meaningful extent every time. What you see with the various "automl" efforts is merely transfer learning, refining of existing models on customer's data. You can learn how to do that in one evening, with no prior DL experience if you're a good coder, and in 2-3 evenings if you aren't a good coder. Point is, it's not a difficult problem, just follow a tutorial.

That's great, that's how smart researchers build practical models, but that's not really "auto-ml", in the sense that there's no hyperparameter tuning going on (as far as I know) and no new problems are being solved. It's just a classifier: anybody can do that with common off the shelf tools and pre-trained checkpoints.

Now achieving state of the art results, solving novel tasks for which there aren't any ready-made solutions, doing DL efficiently, figuring out creative ways to get labeled data, semi-supervised or few-shot methods, etc, that's where it's at right now. And the comp has never been better if you know what you're doing.