Do you have an example? A crawler ignoring robots.txt sounds like a bug, and if you have details I can file a bug with the appropriate team.