> The trend has been to shift as much work possible to the current generation of Data Warehouses, that abstract the programming model that Spark on columnar storage provided with only a SQL interface, reducing the space where you'd use Spark.
I feel like there there are some data professionals that only want to use SQL. Other data professionals only want to use Python. I feel like the trend is to provide users with interfaces that let them be productive. I could be misreading the trend of course.