Since this is from 2022, I’m wondering how “tabular foundation models” could change this. The incredible success of DL we see at the moment comes partially from foundation models learning on a lot of “semi-related” data an “understanding” of the behavior. Something similar has been explored in tabular data as well iirc.

So I would be curious to see latest DL results. On the other hand it is also the case that in most cases where DL based on foundation models is used, specific heavily tuned models outperform the generalistic models. And for tabular data there is a lot of experience how to make it great with tree based models.