This is fantastic, thanks so much for putting this together. In the past I've relied a bit too much on benchmarks here [0] for assessment of warehousing solutions, but it's pretty clearly skewed toward certain query patterns due to operating on a single table (no joins) and limited aggregation.

Using TPC with a wider variety of queries and evaluating on dimensions other than performance provides much better perspective.

[0] http://tech.marksblogg.com/benchmarks.html

Great post!

A couple of thoughts:

1. It'd be interesting to see these benchmarks "under real-world adversity". For example, as data is ingested into the DW, as data shapes change, as concurrency of workloads varies.

2. It'd be great to measure "effort" to reach and maintain this performance.

DW vendors are trained on optimizing for static TPC benchmarks with concurrency of 1, which are useful, but can miss the boat on what users actually experience.

(BQ PM)

I'd like to understand more details about your Presto testing.

The post mentioned ORC files, but doesn't detail how big they are, how many, is the dataset partitioned? Is the data sorted in any particular way?

No Vertica?