"Applebot is the web crawler for Apple. Products like Siri and Spotlight Suggestions use Applebot."
It's crawling things for me at work that I cannot imagine being used for Siri or Spotlight Suggestions, though
Isn't the point of a crawler to crawl nearly everything? Of course they might weigh some things much heigher (recrawl wikipedia much more frequently, for example), but you don't know what might be interesting as a search result until you've crawled it!
How is it possibly going to know whether something is appropriate for those unless it crawls them first?