Hacker News Reader

To store such a firehose stream of data, you will need approximately 0.3gb of storage per one second of data.

This is if you only collect username, timestamp and tweet, excluding any additional metadata such as data center, likes and retweets, not to mention images and videos.

Full calculation here: https://docs.google.com/spreadsheets/d/1BIAguT9Qvy0GK-dalpQf...

View on HN

I believe your calculations are too high by a factor of 1000... 53 bytes/(typical tweet) * 6000 typical tweets/second is 318 KB (or 0.3 MB) per second, not 318 MB.

Yikes, you are totally right. Fixed

Since a couple of years ago the tweet limit was bumped to 280 characters, so I assume the amount is double of what you calculated.

Good point! Fixing my calculation. So it's between 0.3gb (typical tweet is ~30 chars) and 6.8gb per second

Reply on news.ycombinator.com