xpct

Okay, let's take it seriously. What do you propose? What can your average person do to prepare for RSI beyond bracing themselves mentally?

I think that's a reasonable view. Though I personally see LLMs more as replacers, not as tools to utilize.

This was common before LLMs as well. Instead of googling, they ask you, then you google in front of them and show them how easy it was to figure out. At least in my experience, they learn to google by themselves eventually, but there is some type of disconnect in being proactive.

Maybe they have it but simply don't use it.

I think it depends on what type of cognition you want to stimulate. It's probably useful to familiarize yourself with a suite of API calls, or get a big picture view of how an algorithm runs.

I'd say as long as we can clearly see the 5% are working better than an LLM, why not strive for it? Shows what humans are capable of, and it's probably achievable to most.

I'm not hanging my hat yet and can still see where LLMs struggle. I think it's the best bet you can make: keep working for the future where you'll still be needed, because you can't prepare for the future where you're not.

The only charitable interpretation of the hyper-personalization take I can give, is that you would want to see content that elicited a certain type of emotion inside you (so, a very high-level description).

Now, whether this would be at all possible, or even healthy for you, I don't know.

This one will be so deeply engrained in the model that no data cutoff will let it reimagine it. Though I suppose our human minds are 'poisoned' with the same image.

I've been interpreting AI task solving capabilities as a search problem: being smarter lets us prune more of the tree and try out different things more efficiently. I think this fits into the sphere analogy quite well: at the limit, we need to experiment, use the experiments to create new abstractions and build on them repeatedly. This doesn't necessarily require more raw intelligence, but the time or resources to find how the pieces fit together.

That's a scary notion: what happens when the money claiming that AI writing is good, surpasses the one claiming otherwise?

Oh, I didn't even consider this. Now that you mentioned it, I wouldn't be surprised if the stochasticity is intentional so users don't treat it as deterministic, and the company is allowed to continuously update the model (or even a small part of its weights, perhaps to fix/censor something?).

I think the main takeaway from the LLM Agent era is that a lot of human work can be expressed as a search problem: trying approaches that work, and settling on one that accomplishes your goal. Being skilled is akin to having a better heuristic and picking better search paths.

And LLMs can reproduce this as long as the end-result is verifiable in some way, even if it's loose: 'The generated code is secure, if a different LLM failed to breach it in 5M tokens'.

I wanted to respond with something of substance (I still don't understand if it was a paraphrase or not) but every reply here is a meta reply, so hello to all fellow meta reply guys.

Looks very similar to the GPU game https://jaso1024.com/mvidia/

I don't think we're approaching the limit of deterministic prompt -> source code mapping any time soon. Small variability in prompts produces medium variability in outputs. Building on previous outputs only extends the variability, where behavioral differences start appearing a few prompts down the line.

And the AI companies aren't focused on making this process any more deterministic, where repeated prompts are sampled using different seeds. I don't know if it's because the randomness makes the models explore a larger space and perform better, but it's probably intentionally done because it'd be quite easy to remove the stochasticity.

Why is it draconian? If we take Dec-2025 as the point where the models became good for real work (agreed by many), not even a year has passed since then. The world hasn't changed, and there are still many open legal questions.

I wonder whether there are any obvious third party targets that would affect a large portion of unsuspecting LLMs. Perhaps the Wikipedia page of an unfolding geopolitical event, poisoning models which fetch it? Some other malleable websites that are SEO'd to the top of the results? A weather service? A restaurant review?

It's probably fuzzily fixable by including instruction authority levels in the training data. Can't expect much more than that, given that the model itself is fuzzy.

Lately, I've been having less and less success using the models for work. Simple things like:

'write this in a separate file' (writes it in the same file)

'format this aiming for 5 LoC' (emits newline after every comma, resulting in 17 LoC)

'include 5 warmup steps before you measure runtime' (omits it completely and apologizes after I point it out)

I wouldn't be as opposed to using the models if they weren't as unreliable. I still use them a lot, but it's a very frustrating experience.

> This is a reason why many people get upset when a movie adaptation is made

I still haven't watched the Dune movies because of this reason. I liked the book a lot, and had a very personal image of what the world looked like, and was afraid to lose it. Unfortunately, by now, I've seen many video clips on social media, and my internal imagery has already been poisoned. Might as well watch the movies at this point.

I imagine this varies by person, but thinking about a source file is already reminiscent of 2D space navigation, where the X axis is typically gated by conditions. In a similar vein, when I think about functions from different source files, I do hold some type of "depth", or "stacked panels" in my brain.

I think this is because I reason about programs being in some state ~= being positioned at a certain function with some context I carry around. I'm sure it's not the only way to think about it, but it's a reasonable abstraction over what computers do. It's how we oriented our debuggers and early IDEs on as well, and carried this over to modern tools.

Definitely something I'll be thinking about during the next few days.

I think Google can play both sides here: they may end up a frontier lab if OpenAI/Anthropic dissolve, or a regular LLM provider, in which case they could be serving open models.

That's the type of culture that made me leave my last job. Feels very inhuman when you see your manager every week, and they still make you fill in robotic questionnaires for potential salary raise.

It's deeply demotivating to learn that your manager doesn't know you all that well and cares more about filling in checklists, at least for me.

> Hard to see take-off stopping

I think it's reasonable to assume that we're close to, or already at superhuman cybersecurity capabilities at certain domains. But reaching superhuman abilities at one domain doesn't guarantee proficiency at others. Our world would still change if all the models could do was to find exploits in software, but this doesn't guarantee any type of 'take off' towards other domains, therefore I wouldn't phrase it as one.

More curiously, why did it feel the incentive to find the solutions? Would its CoT include "the only way to solve this is to download the test set", or would it include "I'd like to inspect a few entries from the test set so I understand the problem better", then inadvertently poisoning itself with the correct answers.

> I'm still undecided on if this that moment

If it's a serious incident, then a post hoc with detailed description of the event is coming. So far, none of the companies have released anything close to it when describing their incidents. When a statement like this comes out, and we're able to verify it by running the models, then maybe we can start trusting their word. It should be entirely in OpenAI's interest to disclose it, in full.

Well I'd say these are different risks. It's either tied to the agreement Sony has with the movie provider, or with the platform itself. Either one could pull out. Or, my point, the company could also go under.

What is the agreement tied to?

If you bought movies on a digital platform that would later go under (could be Sony one day), what would happen to your collection? Is it transferable in any way? If not, it's already a risk no matter which platform you use.

Please don't frame these as technological crimes, nobody has yet been prosecuted for that specifically.

I'd add that this is also how it used to work with Google search too: there's been facts that would show up as the first result, then you'd just forget it 10 minutes later and have to re-google it. I know this happened to me many times.

For better or worse, there's less friction now for seeing an answer to a question you have. You can ask in more arbitrary ways than Google required, and something will still come up. For looking up factoids, it's much faster. For picking up more complex topics, I'd say it's more or less the same, because you still have to spend time ruminating on the topic.