Kinda like robots.txt, there's nothing preventing a crawler from ignoring it or omitting it and just doing whatever it wants. Slapping in a massive header comment might work.