Hacker News Reader

55 million tweets/day * 140 characters/tweet ~= 14.5 GB/day, presuming a 16-bit character encoding.

That's only about 20 TB for four years worth of data, or about $1500 worth of storage, before compression.

That doesn't seem like it would be a significant challenge for the LOC. I'm more interested in understanding why they would want to use that amount of space to archive Twitter, as opposed to anything else.

View on HN

Combination of low-hanging fruit plus inflated idea of twitter's usefulness (much like other people largely unfamiliar with today's technology culture)?

There's a good bit of metadata (about another 300 characters or so on average) attached to each tweet that makes it useful. Not a huge amount more, but considering the growth curve... yea 20TB is about right.

Reply on news.ycombinator.com