sync_jeff

Understanding why your job performance is changing is critical to both optimize costs and prevent crashes. In this post we show a new visualization timeline features to help engineers find root causes quickly

Thanks for reading!

We provide our own perspective on the high level comparison between Databricks and Snowflake

We built an ML-powered Databricks cluster optimization tool to help remove the tedious work of tuning Jobs clusters to lower costs.

We recently shipped a huge upgrade to our product, Gradient, that can automatically improve your job cluster settings to hit your business goals.

This just came out of private preview where we had awesome results with early testers. We're looking for more feedback and would love to have folks here give it a spin!

Check out our demo video here: https://www.youtube.com/watch?v=i09x6B6mtjg

To get started and try for yourself, see our docs!

We ran a $12K experiment to test the cost and performance of Serverless warehouses and dbt concurrent threads, and obtained unexpected results.

Databricks provides two different ways to monitor usage and costs. In this post we look at the pros and cons of both solutions.

Realities of company and cloud complexities require new levels of control and autonomy to meet business goals at scale

Realities of company and cloud complexities require new levels of control and autonomy to meet business goals at scale

In this blog we run the TPC-DS benchmark to see the cost and performance benefits of Photon and Graviton instances on Databricks. The results are intriguing and reveal a techno-economic struggle!

Our new tool that enables users to programmatically optimize their Databricks clusters.

In this blog you'll learn how to integrate Gradient, an optimization service, into Airflow + Databricks to lower costs and hit runtime goals.

In our blog we discuss our data science work on how we built a system that optimizes Databricks clusters.

Customer blog on how the Sync Autotuner API integrated with their Airflow infrastructure helped to optimize their ApacheSpark clusters.

We perform a cost and runtime evaluation of Graviton instances for Apache Spark on EMR AWS using the TPC-DS 1TB benchmark.

In this blog we analyze the impact of varying the worker instance size on cost and performance of Apache Spark in EMR using the TPC-DS 1TB benchmark.

We take a deep dive into the costs performance of Databricks's autoscaling algorithm with the TPC-DS benchmark.