except google does respect robots.txt so you do have a choice?