Hacker News Reader

I would go to with elasticsearch, or if you reach terabytes/petabytes of data then look into Google BigQuery there you would get a constant 5-10 seconds response time for those high volume data. BigQuery is a WORM technology, so an append-only system might help you, especially in event-driven systems.

View on HN

BigQuery is interesting. Thank you for the link. I am trying to determine if large quantities of data like this can be analyzed with sub-second aggregation time, targeting a 2-3 second total response time to the user, including network latency. The hypothetical data set would not necessarily be event driven, but of course it is useful to have lower data latency. The aggregation speed is the biggest performance concern for me.

Reply on news.ycombinator.com