Hacker News Reader

Comment by buildbot | Hacker News Reader

No it's the death of the corporate content hosting web - the open web was never about making money with your blog post/irc chat/usenet group/etc content, at least in my opinion.

Let data be free! If someone wants to use it to make money, well, it's open, just like open source. It's still not okay to take open source work and claim it as your own, which is what copyright should be limited to. Stealing a photo or plagiarizing an essay is intrinsically different than just having a copy read by something, be it human or an mechanical process such as training a LLM.

View on HN

Maybe I don’t want my non-corporate art to be a part of some large corporation’s training data.

Once again I have to point out that LLMs are not and will not be restricted to "large corporations". Even if initial training is prohibitively expensive, open base models already exist.

"corporate this" and "corporate that" are handy rhetorical flourishes but they distort the debate.

User-agent: GPTBot Disallow: /

ATMO you shouldn't have to maintain knowledge of what kind of crawler bot exist and having to maintain deny list. It should be the opposite, only expressedly allowed content should be crawled by mainaining allow lists.

You can do the opposite since the inception of robots.txt: User-agent: * Disallow: / and then whitelist google bot and whatnot. Most of the web is already configured this way. Just check robots.txt of any major website, e.g. https://twitter.com/robots.txt

The Allow: directive was an extension to robots.txt added later.

That was my gut reaction too, but presumably unless it becomes regulated, at least some competitors to OpenAI won't respect any robots.txt and thus any open content might be training data.

User-Agent: <new technology category>Bot

if you want to give away your data for free, do it, but speak for yourself, not everyone is in position to be able to do it. Just like you can't force other people to give away their software (or anything else, really) for free if they choose to charge for it.

Reply on news.ycombinator.com