Ask HN: Will we see a license explicitly excluding AIs from scraping content?
In addition, should we make an equivalent to robots.txt or include some AI-related rules into robots.txt?
Google won't separate these things out, you get ranked high in the search engine _and_ your content goes into the AI machine, or you get neither.
(1) indexing public content with the purpose of referring to the source in a search result and
(2) ingesting content (no matter the license) to train models that can reproduce smaller or larger chunks later on.
Google and others might be doing this with data from their crawlers already, but that is besides the point. I think these two uses of publicly available data is fundamentally different and that a distinction should be made.