Yes, it does automatically keep a cache from when it indexed the site. I have it set to not automatically recrawl sites, so the cache is from when I added the site.