fangjin

Druid founder @zedruid writes a (satirical) benchmarking blog post about Druid vs Clickhouse and Rockset. The results are technically valid, but should you ever trust a benchmark?

The databases listed above, to the best of my knowledge, are commonly used for dev ops metrics data and share similar terminology. Druid on the other hand, draws much of its terminology from the OLAP world. As cheddar clarified in his post above, the benchmarks for Druid are misleading as it is not an apples to apples comparison (I suspect the benchmarks for ES also suffer from this problem). A single Druid event may consist of thousands of metrics.

I think we're using very different terminology here. An event in our world may contain thousands of metrics as part of the same event.

Here is a source from 2015: http://www.marketwired.com/press-release/metamarkets-clients...

You can also find additional information that folks have been willing to publicly share on scale and use cases here: http://druid.io/druid-powered.html

While Druid was initially built for batch loads, the architecture has evolved substantially as the project has matured. Today, Druid supports exactly-once streaming ingestion from Kafka, and large production deployments routinely stream millions of events per second into Druid.

FWIW, Imply repackaged Druid in such a way that should make it much much easier to set up and evaluate. We've been porting our docs over to Druid for 0.9.0: http://imply.io/docs/latest/quickstart

There's also a production-ready docker distribution: https://hub.docker.com/r/imply/imply/

Druid has a SparkSQL connector: https://github.com/SparklineData/spark-druid-olap

Druid's main value add to the data infrastructure space is around power user-facing data applications at scale. The queries it is best at are OLAP/business intelligence style queries. It isn't really designed to be a general processing tool such as Hadoop or Spark. The open source data space is very complex, and there are many different solutions targeted towards many different use cases. Druid is better than other solutions at some of these use cases, and worse than other solutions at others.

I wrote my interpretation of the current open source data landscape here for anyone interested: http://imply.io/post/2015/11/04/big-data-zoo.html

Druid committer here. We spent a lot of time in the early days on making sure the system worked (at scale) and now we're spending more time to make it much easier to set up and manage.

Druid (http://druid.io/druid-powered.html) is another option for similar workloads. Druid is a community-led open source data store used by many technology companies at very large scale. Comes with multiple visualization/open source applications, SQL interfaces, Grafana extensions, and a community to help with issues.

Hi Evan, the user groups (https://groups.google.com/forum/#!forum/druid-user) are very active and if you post your issue there, I"m sure someone could help.

The logo draws its origins from mathematics :) http://mathworld.wolfram.com/Implies.html

Druid does pre-aggregation (roll-up) of data at ingestion time and is also used at scale (30+ trillion events, ingesting over 1M+ events/s) by numerous large technology companies: http://druid.io/druid-powered.html

At a high level, the architecture seems to be very similar to Druid, although the Pinot team has yet to release more technical information about the project. Druid is farther along in terms of development, community, adoption, and scale. It is curious why the team choose to rebuild the architecture versus just using Druid.

Here's a direct link to view the paper online in case people don't want to download it: http://static.druid.io/docs/radstack.pdf