So are they going to try and IP gate them or trust that AI companies that literally stole the info they used to make the base models will now respect robots.txt entries?
Every one likes net neutrality when the one is benefitting from it, yet the one immediately jumps at the opportunity to break net neutrality on their services if it allows to increase profit by price discrimination (which may take a shape of extracting a rent from some subset of consumers like seems to be in this case) .
AI scrapers aren't a subset of the users/readers/audience of a website, they are parasites to society that harm their prey by causing huge costs.
why? Aren't the AI scrapers working under a company which creates value therefore exists in the first place?