Can someone give me a real-world example of a scenario where they actually need a time series database, like an example query with the business use case / justification? Just super curious.
Not the archetypal time series use case, but TimescaleDB is still really useful.
TSDB's compression means we can store a huge volume of data in a fraction of the space of a standard Postgres table. You can achieve even better compression ratios and performance if you spend time designing your schema carefully, but honestly we didn't see the need, as just throwing data in gets us something like 10:1 compression and great performance.
TSDB's chunked storage engine means that queries along chunking dimensions (e.g. timestamp) are super-fast, as it knows exactly which files to read.
Chunking also means that data retention policies execute nearly instantaneously, as it's literally just deleting files from disk, rather than deleting rows one-by-one - millions of rows are gone in an instant!
And best of all, this all works in Postgres, and we can query TSDB data just the same as regular data.
All that combined easily justified the decision to use TSDB - and if you're familiar with Postgres, it's actually really simple to get started with. Really, we'd of needed a business justification not to use it!
Much love for the TimescaleDB team!
https://twitter.com/ongresinc/status/1451194886795849742
(Also, thank you GordonS for the kind words!)
We wrote more about this for an earlier Timescale blog post: https://blog.timescale.com/blog/how-everactive-powers-a-dens...
That is where a time series DB is very handy
1. Render job logs, combined with host metrics to determine if any systems are being under utilized. 2. Intranet network usage for monitoring purposes. 3. General monitoring of log events, system metrics, etc with alerting based on specific query criteria. 4. Temperature / Humidity monitoring and trend analysis.
Basically if there are things that happen somewhat often (http logs, system logs, host metrics, power usage, temperature change, etc) and you want to analyze that data, and one of the factors in your analysis is time, then time series databases are for you.
In the end each TSDB has different trade offs, and there are a number of considerations to make, and how you weigh them will change a lot depending on scale. If you are writing 1-20GB/Day vs 1-20TB/Day, you will have different considerations. If you have relatively simple queries vs complex queries with complex joins on other data you will have different considerations. In the end you need to have some idea what your data is, how much of it there is, what the growth looks like and what your write/query usage patterns are going to look like. Insert / Query performance isn't everything, unless for your business it is.
Large scale infrastructure monitoring?
If you run a data center with 10K machines in it, whitebox monitoring of these machines and what runs on it generates tons of timestamped data.
These time series can be used to inform an automated alerting system (eg using trends to forecast bad things before they happen).
They can also be analyzed in batch mode to figure out how to optimize many things (power / cooling / workload assignment / etc ...)
FRED (https://fred.stlouisfed.org/) is a great resource for publicly available economic time series data.
This is a great post to give you some talking points:
https://blog.timescale.com/blog/what-the-heck-is-time-series...
I also love this recent one we did with some non-standard time-series data that the NFL provided! Really fun working on that data set.
https://blog.timescale.com/blog/hacking-nfl-data-with-postgr...