It seems that ChatGPT can reference Reddit data ~30 seconds after it's posted | Hacker News Reader

It seems that ChatGPT can reference Reddit data ~30 seconds after it's posted

old.reddit.com

This is an interesting case, I think the sequence of events was this:

- Someone started a thread related to this topic 3 weeks ago

- Bing indexed that page a while back

- OP found this thread, and then posted on the same thread, today

- ChatGPT used a Bing search tool, and comes up with just the right search query such that that very same reddit thread is one of the top returned results

- ChatGPT links back to the user the thread they just posted to

Out of curiosity, if you use Google for the same query (at least for me) that thread is still the 2nd top result, so if a Google AI was asked in the same way it might be able to do the same thing.

So then I wondered, is it that this is random luck to do with the exact phrasing of the query? So I asked Claude to rephrase the query, it gave me "Wi-Fi 6E frequencies used by Apple Wireless Direct Link" - and the same reddit thread popped up again as the #2 result.

I think basically this is attributable to

- search engines being relatively good at finding the things people are asking for

- reddit results probably having naturally boosted rankings in search results, and

- the user possibly having gone through a similar process/flow (either via a search engine or reddit search), using the same types of routes to (and signals about) useful sources of information, to find the same niche recent reddit thread about this topic, as the LLM did

Is ChatGPT doing this via function calling (asking Bing), RAG or another way? Any model capable of online learning yet? Aka using new data and conversations to fine tune/train while doing inference, like humans do? I have seen some claiming it does, but do they really as it seems a difficult problem as our human long term memory seems to involve dreams and other factors to do this which LLMs don’t have. Also, while I believe it is needed for a next step, we humans aren’t bothered to learn and believe crap like flat earth and cursing and unsavory jokes; LLMs that are not local cannot (I don’t agree but I see the problem with lawsuits for big corps) be permitted to do the same.

Anyway ; is that possible yet? Or what is the state of the art? Cannot see it really going anywhere unless LLMs can internalise new info into their NN while online instead of in months of offline training.

It asks bing gpt,becomes this reflective and self aware,gets a result, then dies like a wave on a shore.

It has been able to do that for a long time really. Is there anything new / added to that this time (I have no access to searchgpt yet).

This makes sense. It searched the web to find the comment.

What index would you expect to have Reddit comments 30 seconds after they're posted? This is more likely pulling directly from Reddits API than it is pulling from web searches.

In this case it seems the search engine had indexed the thread/question, which was posted previously (making sense it was already ranking in search results), and ChatGPT followed that link and a found the answer.

If I was Google I'd be shitting my pants.

Reply on news.ycombinator.com