nhourcard

I think this is a little bit harsh; you are pointing to one specific query that does not work out of a dataset of 1.6 billion rows exposed live to the internet and could sustain tens of thousands of users concurrently. In any case, we're grateful for the cto of clickhouse to point those things so we can improve the product further. But generalisations such as "the demo does not work at all" is not a fair comment, nor beneficial IMO.

yes correct - although Clickhouse is more of an OLAP database. Timescale is built on top of Postgres, while QuestDB is built from scratch with Postgres wire compatibility. You can run benchmarks on https://github.com/timescale/tsbs

Hi tgv, curious to hear why you picked NoSQL for your time-series use case?

It depends on your use case. For example, Postgres will hit limitations for large streams of time series data on the ingestion side, and the standard SQL language of Postgres may make it challenging to navigate a dataset by time: sampling the data, time intervals, time-series joins (ASOF join), and these kinds of things are not easy or possible to do on Postgres.

Why not combine Postgres for OLTP workloads alongside another database for fast analytics/time series, optimised to deal with high throughput ingestion and low latency queries?

NB: I am one of the co-founder of an open-source time series database (questdb)

QuestDB | Developer Relations engineer & Growth engineer| Remote | https://questdb.io/

We're building an open source time-series database focused on performance and simplicity.

Developers rely on QuestDB as the analytic backbone of real-time systems ranging from FinTech to machine learning, IoT, and application monitoring. Fortune 500 companies such as Airbus and Yahoo deploy QuestDB for large-scale, data-intensive production systems, some of which serve close to a billion users.

Our open source repo has reached 10k GitHub stars and we have raised $15m in capital to date from YC and leading venture capital funds.

We hire talented and passionate people who share our mission to empower developers to solve their problems with data. We are building breakthrough technology to power the infrastructure of tomorrow.

We're looking for a developer relation engineer and growth engineer:

- Our career page: https://questdb.io/careers/

You can send an email directly to careers@questdb.io

Thank you for sharing the post! I’m nic, ceo of questdb, your feedback about our implementation is greatly appreciated

You're right, this post focuses on filter queries with the WHERE clause. Aggregates are also pretty fast, a live demo is currently available with three datasets, one of them being more than 1.6 billion rows: https://demo.questdb.io/ On this you can also perform a join, in particular ASOF joins (where timestamps do not exactly match).

NB: The launch of this demo was done some time ago on HN: https://news.ycombinator.com/item?id=23616878

QuestDB | Python/cloud engineers, UX Designer, Solution engineer, Core Database Engineers | 100% Remote |

QuestDB (YCS20) is building the fastest open source time series database. We hire talented and passionate people who share our mission to empower developers to solve their problems with data.

https://questdb.io/careers https://github.com/questdb/questdb

QuestDB | Open source time-series database | Remote | https://questdb.io/about-us/

QuestDB is a fast open source SQL time-series database, with users including industry leaders such as Yahoo and Airbus. We are obsessed about performance and built the entire stack from the ground up with zero dependencies in the code.

We are hiring developers in tests, customer success engineers, core database engineers, backend engineers (python) and developer relation engineers

https://questdb.io/careers/

QuestDB | Developer Relation Engineers, Python Backend Engineers | Remote

QuestDB (YC S20) is an open source time series database. Find more about us here: https://questdb.io/about-us/

We're hiring Developer Relation Engineers, Python Backend Developers, Core Database Engineers (zero-GC Java, C++) and Customer Success Engineers. https://questdb.io/careers/

QuestDB | Backend engineer (python) | Remote| https://github.com/questdb/questdb

We help developers handle explosive amounts of data while getting them started in just a few minutes with the fastest and most accessible open source time series database.

We're hiring a python backend developer to build a SaaS solution from the ground up.

Link: https://questdb.io/careers/senior-backend-engineer-python/

QuestDB | remote | https://github.com/questdb/questdb

We help developers handle explosive amounts of data while getting them started in just a few minutes with the fastest and most accessible open source time series database.

We're hiring across engineering, developer relations, customer success, talent acquisition.

Link: https://questdb.io/careers/

QuestDB (YCS20) is an open source time-series database. We're growing the team and hiring core database engineers (low-latency java/c++), cloud engineers, front-end engineers and developer relation engineers. https://github.com/questdb/questdb

At QuestDB we chose zero-GC Java for 80% of the code base, which resulted in superior performance on ingestion compared to the alternatives.

QuestDB (YCS20) | several positions | Remote QuestDB is an open source time-series database built from the ground up in zero-GC Java and C++, without a single dependency in the code. Our users, such as Verizon, deploy QuestDB to make time series analysis fast and convenient in fintech, Industrials, application monitoring and machine learning.

https://questdb.io/careers/

Developer Relations Engineer

Senior Cloud Engineers

Software Engineers, Backend

Software Engineers, Frontend

QuestDB (YCS20) | several positions | Remote

QuestDB is an open source time-series database built from the ground up in zero-GC Java and C++. Our users, such as Verizon, deploy QuestDB to make time series analysis fast and convenient in fintech, industrials, IoT, application monitoring and machine learning.

https://questdb.io/careers/

Developer Relations Engineer

Senior Cloud Engineers

Software Engineers, Backend

Software Engineers, Frontend

This is something we can eventually do, it seems that m3 position themselves as long term storage for Prometheus first and foremost.

This is something we should try :)

Indeed! One can think of time series data as consecutive events happening such as logs, where the series is going to give the historical evolution over time.

Thanks for the kind words! QuestDB co-founder here - one example in fintech where you are polling multiple end points simultaneously for market data acquisition. You can think of multiple financial exchanges sending the data, but the latency varies for each venue. This is a great article written by one of our contributors about ingesting data from multiple crypto exchanges at the same time [1]. Many firms in financial services and fintech use QuestDB [2]

Another would be a manufacturing firm with thousands of sensors sending data to the database continuously, latency and delivery mechanism mean that timestamps often arrive out of order. There is a case study we did with CNC machines manufacturer DATRON [3].

Tracking moving vehicles (say cars, ships, drones, planes) on a map where the two primary axis are time and space - if you want to see the position of those vehicles in the past, you want to have the timestamps ordered.

Finally you can think of typical devops/monitoring/alerting use cases. Verizon uses QuestDB to monitor metrics for autoscaling decision to their Vespa engine that provides search, recommendation, and personalization to their users [4]. Here you want the series to be ordered by time.

[1] https://levelup.gitconnected.com/tracking-multiple-cryptocur...

[2] https://www.tradersinsight.news/ibkr-quant-news/optimizing-t...

[3] https://questdb.io/case-study/datron/

[4] https://questdb.io/customers

QuestDB (YCS20) | several positions | Remote

QuestDB is an open source time-series database built from the ground up in the name of performance in zero-GC Java and C++.

https://questdb.io/careers/

Developer Relations Engineer

Senior Cloud Engineers

Software Engineer, Backend

Software Engineer, Frontend

The death of moore's law made us wonder - there is so much effort trying to optimise hardware, but less emphasis on making software more efficient. Our view is that there is a lot to do with regard to software efficiency to mitigate the limitations in hardware progress... See the company we founded in my profile, this was one of the drivers to build it

QuestDB the fastest open source time series database built from the ground up to offer breakthrough performance for real-time analytics using SQL. We are a remote-first company based in London and backed by leading venture capital firms and Y Combinator.

Roles: Senior Cloud Architect: https://questdb.io/careers/senior-cloud-architect/ Software Engineer, Backend: https://questdb.io/careers/backend-software-engineer/

Best of luck Aaron! Really enjoyed our conversations at YC

Thank you for posting this benchmark, great addition to the TSDB community. We look forward to benchmarking our SQL time-series database QuestDB[1] versus both using the TSBS framework soon!

[1] https://questdb.io/

"Columnar databases aren’t new, so why are we building yet another one? We weren’t able to find one in open source that was optimized for time series."

This is the direction that QuestDB (www.questdb.io) has taken: columnar database, partitions by time and open source (apache 2.0). It is written is zero-GC java and c++, leveraging SIMD instructions. The live demo has been shown to HN recently, with sub second queries for 1.6 billion rows: https://news.ycombinator.com/item?id=23616878

NB: I am a co-founder of questdb.

We did YC remotely this year. The flexibility from being remote was a real plus. Less time spent on SF related admin, that can be utilised to connect with more founders. Obviously you lose out by less meaningful connections that are made face to face.

One cool aspect was to talk to companies building out the remote tools of tomorrow and see what the future could en up looking like.

The value from YC still applies in a remote environment. YC demo day remotely is actually probably even better for founders and just as effective.