matthewhelm

I wholeheartedly agree. When I worked at Shopify, we had to hand-roll our incremental data models using Spark, and the complexity of managing deep DAGs made tasks like backfilling and refactoring a huge pain. Tools like dbt and SQLMesh face similar challenges.

The chaos of existing approaches was a large part of what drove me to join Materialize. With Materialize, you can use dbt on “easy-mode”, while Materialize handles incremental logic, removing the usual headaches around processing time and keeping everything up to date within a second or two.

I recently gave a talk at Data Council about this unlock, it’s total magic: https://youtu.be/pLb5sFZ7nWw

For anyone interested, my colleague Seth also discussed this in a recent blog post: https://materialize.com/blog/migrating-postgres-materialize/

Congrats on the launch! I work as a data engineer at Shopify designing data models and building data pipelines for our merchant-facing analytic products. I’ve learned a lot about model design from Kimball, but (as you mention) found it vague in some critical areas. I’ve ordered a copy and am excited to give it a read.

"$4 billion in cash and approximately $12 billion worth of Facebook shares"