I'd stand by the bitter lesson ( http://www.incompleteideas.net/IncIdeas/BitterLesson.html ) here. of course it feels better to do something clever, finetune models, do clever agents or novel algorithms instead, ... only to become beaten by the next larger scaled general model.