Hacker News Reader

Comment by CamelCaseName | Hacker News Reader

Can someone explain what value new Reddit data has for LLMs?

Presumably the entire 2005-2023 (up to March) dataset which is (was) freely available on Pushshift / torrents is enough?

Also, not that Sam Altman is unbiased, but he did say that social media (Reddit, Twitter) were not super important for developing LLMs (But of course, that's exactly what he would say if they were useful!)

View on HN

Depends what you're training it on, I suppose. If you're just wanting to train something to generically talk then sure, a slightly outdated data set is fine. If you're wanting it to be able to talk about recent things then it'll, naturally, not be fine. It's not so much a problem right now, only being a couple of months out of date, but once that turns into years it'd become more apparent.

But sure, there are plenty of other data sources as well. On top of the problem of not wanting to train your data on content you know is littered with bots.

I'm now envisioning a sci-fi future where the AIs are an embodiment of xkcd #606. Far better than humans at mundane tasks but with significantly outdated cultural knowledge.

There are humans today that refer to Africa as the 'third world' even though it's a continent with countries more developed than some US states or European countries.

Significant outdated cultural knowledge seems the norm for humans too.

As someone in a US state with blackouts to our power grid, I'm wondering what more developed things you might be referring to and I might be missing out on.

Are they using "third world" in its original sense of non-alignment?

https://en.wikipedia.org/wiki/Third_World

I didn't even know it was available on torrents.

Reply on news.ycombinator.com