Yes, I have noticed that creating a brand new dev domain with crawler blocking norobots file, it is not found on any search on Google, until I open the dev url in Chrome, then bam! watch as their crawler starts trying to search through the site just from opening the url in Chrome.
This is why I never use Chrome. They scrape the Google Safe Browsing sent from chrome browsers and just do not care about privacy.