thread on HN from when common crawl was announced, interesting info there: http://news.ycombinator.com/item?id=3209690