Agentic crawlers are worse. I run a primary source site and the ai "thinking" user agents will hit your site 1000+ times in a minute at any time of the day