millions of rows per device. After my where conditions
I know I can have around 100000 devices
That's going to be a fairly heavy bit of data; my wild-guess estimate is around 1TB for 100k devices with 1M rows each of 12 bytes (4-byte ID and 8-byte timestamp).Is there anything special about "specific_date_time_value"? An efficient architecture may be able to use that to avoid considering every row every time - for instance, using a queue to surface the "next" event easily.
So far I’ve tried modeling this in: - clickhouse - Postgres - mongodb
How did those turn out? What worked and what didn't?You've already collected a lot of info, sharing it will help!