Hacker News Reader

Comment by fasterik | Hacker News Reader

>Let’s look at what Microsoft is doing right now, as an example. In the last couple of weeks, Microsoft updated its general terms of use to prohibit scraping, harvesting, or similar extraction methods of its AI services. Also in the couple of weeks, Microsoft affiliate OpenAI released a product called GPTbot, which is designed to scrape the entire internet. And while they don’t admit this publicly, OpenAI has almost certainly already scraped the entire non-authwalled-Internet and used it is training data for GPT-3, ChatGPT, and GPT-4. Nonetheless, without any obvious hints of irony, OpenAI’s own terms of use prohibits scraping.

I don't understand why this demonstrates hypocrisy. There is a big difference between crawling the publicly accessible web (which legitimate search engines do all the time) and scraping an authenticated web application or API.

View on HN

The hypocrisy stems from:

1. OpenAI (etc.) scrape the public web to train and build their models.

2. They use these models to sell subscriptions (make money). None of this goes to the creators of the data used to train the models.

3. They deny others to do what they have done themselves.

If you compare to, say, search engines scraping the public web:

1. Search engines scrape the public web to create their search indexes.

2. They use these indexes to provide search results and sell ads around them. Importantly, these search results direct people to the websites that they have scraped (much of the time), offering opportunity for them to make money.

If you publish something on the open web, you're making that content available to anyone who can send a GET request or use the Internet Archive. On the other hand, if you're making a private application where users create an account and agree to a terms of service, you're allowed to define what kinds of requests are or aren't legitimate. I still don't see any hypocrisy here because the two use cases are fundamentally different.

> If you publish something on the open web, you're making that content available to anyone who can send a GET request or use the Internet Archive.

What about "by continue using this site you agree to this TOS"?

What if you don't have rights for published content?

What if you make your content free and opensource, but don't want big greedy corporation to use your content to train AI?

What about author rights? If I publish my painting does that mean that any corp can sell t-shirt with this content, because "content available to anyone who can send a GET request or use the Internet Archive"?

>What about "by continue using this site you agree to this TOS"?

Terms of service determine when a provider will refuse to service a user's requests. If the website responded to the request with the requested content, TOS is a moot point.

All of the other examples are covered under copyright law. Whether or not copyright has been violated depends on whether the training of AI models falls under fair use. That remains to be decided in the courts, but I think there is a plausible argument that an AI model counts as "transformative use" and wouldn't be a violation of copyright.

Or that while Microsoft is an investor in openai, they do not control openai

If SDGs learned us anything is that companies are responsible for to scrutinize the company they are investing in.

Of course this is not on the level of child labor, nor environmental pollution.

Reply on news.ycombinator.com