How are you dealing with the fact common crawl updates their data much less regularly than commercial search engines? And that each update is only a partial refresh?
Edit: And I will say your site design is very nice.
Edit: And I will say your site design is very nice.