Hacker News Reader

Comment by alkonaut | Hacker News Reader

I worry (And I assume this is either dismissed or a core concern in the field because it's so obvious) that the coming generations of models will use a lot of input from existing models.

Will we be chasing "untainted" input (pre-2022 data) just like we chase low-background steel (pre-WW2) for sensitive machinery? Should we be "tagging" generated data NOW when it's published in large chunks like images, long texts?

View on HN

First thing I thought reading your comment was "this guy's repeating the low background steel analogy that was posted the other day". I don't know if that's actually true, and it doesn't matter, my point is the internet is already an echo chamber. GPT is trained on reddit and HN and other forums where people say the same stuff over and over again anyway. It's not like the pre-gpt internet is some pristine pool of new ideas. As long as someone somewhere is injecting some new stuff in, and they will be, it doesn't matter that there will be more automatically generated crap, there already was

Ignoring the effects of volume is a common and fatal mistake. "This effect already exists" is not incompatible with "Multiplying the impact of this effect by orders of magnitude will cause it to become a massive problem."

Or, put another way: A slightly degraded signal-to-noise ratio isn't really a problem, but once it degrades far enough, the whole stream becomes impossible to decode, and worthless.

My Information Theory is rusty (and wasn't too strong to begin with), but didn't Shannon prove that we can decode a signal given arbitrarily bad noise, but that the more noise, the lower the channel capacity?

Yep, pretty much. But therein lies the problem: filtering out garbage writing is something done by humans, whose limited time, attention, and effort all make for a pretty tight bottleneck in the first place. Degrade that channel capacity, and our ability to find texts of value becomes pretty close to nil.

edit: Also, when the noise comes in the form of competing signals rather than random noise, that becomes much, much more difficult, but we're also outside my area of expertise at this point.

As I understand it, LLMs use temperature scaling to optimize the variability in the tokens they generate so the output appears "good", somewhere between getting stuck always saying the same thing and saying nonsense. [0]

I wonder if more output being used in the training data will result in a shift to higher temperatures over time to keep variety in the output.

[0] Discussion the other day: https://news.ycombinator.com/item?id=35131112

I literally made a Tumblr called https://lowbackgroundsteel.ai the other day :-)

I love that you use the same Steel analogy I came to. Data sets from pre-2023 are going to be useful in determining the validity of future data sets - if that is possible.

What's the use case for such validation?

Why didn’t we do that before AI? If that’s useful now, surely it would be useful to know what knowledge pre-dates humans learning and repeating it?

Well, isn't this pretty much the definition that historians use for "primary sources"?

Don Hertzfeldt's Simpsons couch gag seems like an end result of AI training AI. https://www.youtube.com/watch?v=6i2l-LQ-dXI

I mean, the singularity proponents see this as the golden grail, with the condition that N+1 AI trainee is slightly smarter then the N AI trainer.

I work in a field where the training data is extremely sparse because labeled data is incredibly time consuming and labor intensive to come by. There’s ways to automate labeling, but I don’t really trust those methods fully cause I’ve seen how the sausage is made, so to say.

Everybody and their dog is trying to do machine learning, but I question how useful or real the results are because I know that the data underlying the models is of questionable quality. I don’t know what’s next for the field, but it’ll probably be some high profile disaster.

It is a safer bet that someone will hook a neural net up to a camera on someone's hat, train from reality, then re-derive art from that. Then AIs will use the same methods as humans to maintain their grounding in reality.

The problem with the current crop of AI is they pretty clearly don't understand that they are taking a 2D projection of a 3D concept. That'll be fixed and there is a good chance it'll be fixed with video footage because that is how humans do it.

The immediate benefactor of tagging non-ai generated data would be OpenAI.

Sure. But it would be a lot easier to tag output like images as "AI generated" in a small handful of generators like StableDiffusion and similar, than making the opposite and telling people to tag their non-ai-generated content as non-ai-gemerated.

Also people who want to make a quick buck by selling AI-generated data and marking it as non-AI. Sounds super cheap and high ROI.

I would expect that smart people generate data with GPT that is more and better than smart people alone. I really don’t see the issue.

Especially since smart people + GPT + empirical data (think GPT plus surveys and sensors) will be a HUGE source of intelligence.

Will not-so-smart people be banned from using GPT?

they will have to pay more. And I'm already broke :(

> Will not-so-smart people be banned from using GPT?

If we're consuming information filtered and noise reduced by very smart AI's how much does it matter?

I responded to (from the parent comment):

>> I would expect that smart people generate data with GPT [...]

Also:

> filtered and noise reduced by *very smart* AI

Are you sure you know what GPT is?

Of course no! Only people that can't afford ChatGPT will be out of luck.

I'd agree if it weren't the case that ChatGPT is the fastest growing free AI service in the world. Indeed I cannot think of a single worse example to "The future is already here, it's just not very evenly distributed" - everyone who wants to use ChatGPT can, what more could you expect?

What I only can hope about is the natural selection of useful data throughout time. Garbage is all around, people generate it on mass scale with modern technology nowadays, but the useful portion may survive the test of time, prevail. Those using 'good' data over 'bad' can gain benefits that could lead to the not self harming population using good data, suppressing bad ones. Mostly, of course.

Low-background steel is actually pre-nuke tests. That's what they scavenge sunken ships a lot for.

Yes it should really be "pre end of WW2" (The first nuclear test was 1945).

there's active work on watermarking LM outputs. here's one example:

https://huggingface.co/spaces/tomg-group-umd/lm-watermarking

A lot of AI training data is bootstrapped from weaker models.

I am guessing in the future they would just run simulations to generate the untainted training data pre 2022.

...oh wait.

Reply on news.ycombinator.com