We use daily cron jobs that fetch from various RSS urls for blogs, and use the Twitter API. We also have data contractors that do some manual work to aggregate the content.