Hacker News Reader

Comment by majormajor | Hacker News Reader

Delta is a fairly recent entrant, and when some of my coworkers evaluated it, it didn't really seem as compelling as Snowflake or BigQuery. Databricks was something our data science department liked, but it didn't have a compelling sell to our (much larger) analytics org.

View on HN

Yeah, we moved away from spark for a reason (you kinda have to babysit it or it’ll crash on anything actually big data). Snowflake took care of things much better than that. Their lake offering is like the worst of both worlds.

Spark allows you to distributed model training though, which is really nice when you need it.

> Databricks was something our data science department liked, but it didn't have a compelling sell to our (much larger) analytics org.

So they’ll just have to suffer running their non-SQL workloads locally on corporate issued laptops? Or they’re not really going to do data science at all?

Who's they here? The non-data-science analysts? They were running SQL in the cloud on other products. Non-sql workloads were the domain of data engineers and data science in that org.

The data scientists were using Databricks, but data eng was instructed to try to replicate a few core pieces because it was $$$$ as far as "a way to run notebooks and such" went.

I don't think Delta is meant to compete with Big Query, although it's been a long time since I've used Big Query, I recall upserts not being something you were able to do within BQ and that's essentially the main selling point of delta.

Reply on news.ycombinator.com