is there a place with a list of aws servers these companies can block?
https://platform.openai.com/docs/bots
It does not document how the bot will respond to things like 429 response codes. It does not document how frequently their bots may crawl a given domain/url. It does not document any way to get in touch about abusive crawler practices.