Hacker News Reader

Hacker News

Comment by nylonstrung | Hacker News Reader

Parent Full thread

nylonstrung

It makes no sense that the LLMs could be solving problems based on prompts given by individual scientists

Are they implying that chat history of a single person is being taken an used to skew post-training results?

Why would it only be for Navier-Stokes that we see this having an effect

There are 1 million times more prompts being processed for stuff like furry roleplay than unsolved scientific problems

View on HN

If only we had a technology that could classify the topic, tone, and sophistication of arbitrary natural-language conversations; you know, to sort the proofs from the furries…

It's actually probably the case the proofs are coming from the furies.

Training the LLM on logs might make little sense, but that doesn't mean the LLM can't get access to those logs outside of training. Companies have been gobbling up your data left and right, why wouldn't these companies do the same and then feed it to their LLM for a specific use case?

I think Anthropic et al. would have very little difficulty determining exactly who their users are, so it would probably be trivial to focus any training of this sort on individuals known to be doing cutting edge research, as well as their colleagues, postdocs, grad students etc. Not saying it is happening, but they certainly have the capacity to make a system to skip all the prompts they don’t care about and leave just the prompts they do care about.

Are you kidding? It's clear that frontier labs are desperate to ingest any human text they can find. They're paying people to find rare books and digitize them. They're vacuuming up Reddit threads. And in their own products, they make it difficult to turn off conversation logging unless you pay.

I have zero difficulty accepting that they're using conversation logs, probably anonymized and then chopped so that they can claim it's no longer user data and that it can't be tracked back to a specific chat. And yes, sometimes, a single piece of text is enough to move the needle, especially when it comes to some exotic, sparse topics.

Reply on news.ycombinator.com