Hacker News Reader

You can get a pretty big, random chunk of the public timeline with statuses/firehose or statuses/sample

http://dev.twitter.com/pages/streaming_api_methods

Here is the original paper - http://arxiv.org/pdf/1010.3003v1. The paper claims their code and data is open, but the link is broken and I was unable to find it by browsing their site...which seems to consist of mostly blank pages.

I interviewed who did semantic tweets research at [some university] and said they were given an insane sample from the past x years just by making a written request to Twitter.