Hacker News Reader

We've gotten a lot of passionate feedback in this category, and we actually did a comparison with sort and dist keys in the extended version. They don't make as much difference as people think, these charts would be ~15% shifted with sort and dist keys.

View on HN

Did you guys implement cluster keys in BigQuery as well?

(BQ PM)

No we just did a comparison between Redshift with no tuning, conservative tuning, aggressive tuning. The point was just to assess "how much does tuning matter anyway?"

You are correct that if we wanted to do a full-on "tuned comparison", we would need to implement partitioned tables in BQ, Snowflake and Presto.

(and clustering in BQ! [0])

[0] https://cloud.google.com/bigquery/docs/clustered-tables

That actually sounds like a solid improvement. Do the head-to-head comparisons change with sort- and dist-keys? What was the tuning used?

You can see the whole back-and-forth here:

https://github.com/fivetran/benchmark/issues/1

The tuning was pretty aggressive, some of the sort keys were a little unrealistic for a real-world scenario where you're doing a lot of different queries.

Reply on news.ycombinator.com