bro, since when vc funded ai companies have the courtesy to respect robots.txt?