Didn’t archive.org at some point say they would ignore robots.txt? I wish they would.