1,479 karma · joined May 14, 2013
We got the first news about Mythos in March, so it is likely that it was already close to ready by the time Opus 4.6 was released.
So the actual gap is the time elapsed between March (or April for the official announcement) and whenever Chinese models can match Mythos.
In practice, we seem to be leaning towards the idea that training on a copyrighted book is wrong if used to replicate or paraphrase that same book, but not if used to teach a model how to write better.
And my intuition is that no, you can’t, they are two aspects of the same reality.
Your brain and your whole body exist in time. Even when you are asleep, your body does not flicker out of existence and your brain actually continues working during that time.
Would you be "you" in a different body?
Hylomorphism: body and consciousness are intrinsically linked. The nature of that link is an open metaphysical question.
Virtue ethics: even if LLMs are not conscious, we should not abuse or mistreat them them because cruelty practised on anything trains one's disposition toward cruelty.
Even if you are only interested in getting good results out of them, LLMs tend to work better when they are immersed in a narrative of open collaboration.
Input stream comes in, input stream comes out. The LLM doesn't care whether this happens once a minute or once a year.
> the AI has been able to explore all these possibilities much more comprehensibly, and doing that it found a path, it found a way to the solution.
Finding a counterexample of a mathematical conjecture strikes me as not that different from finding a vulnerability in a complex codebase.
This already happens to some degree on large software projects with corporate backing (Web engines, compilers, etc.), where it is often not trivial to start contributing as an independent individual.
Reasonable people can disagree on whether one approach is inherently better than the other, as ultimately they seem to be optimising for different goals.
> That's a behavior narrowing I introduced for simplicity. It isn't covered by the failing tests, so you wouldn't have noticed — but strictly speaking, [functionality] was working before and now isn't.
I know that a LLM can not understand its own internal state nor explain its own decisions accurately. And yet, I am still unsettled by that "you wouldn't have noticed".
Japanese population is still over 120 million. Forecasts put it falling below 100 million at some point in the second half of this century.
Things will have to change in order to keep population stable in the long term, but the Japanese approach seems IMHO more sensible than that of other countries.
Cohesive democratic societies are fragile.
> Our tests gave models the vulnerable function directly, often with contextual hints (e.g., "consider wraparound behavior").
What happens then is that, for example, the model looks through that particular file, identifies potential problems, and works upwards through the codebase to check whether those could actually be hit.
“Hum, here we assume that the input has been validated, is there any way that might not be the case?”
This is not unique to Mythos. You can already do this with publicly available models. Mythos does appear to be significantly more capable, so it would get better results.
The research discussed here provided models with just a known buggy function, missing the whole process required to find that bug in the first place.
A brain cut from its body and frozen its a dead brain.
It is like a crystal that shows beautiful colours when you shine a light through it. You can play with different kinds of lights and patterns, or you can put it in a drawer and forget about it: the crystal doesn’t care anyway.
Bundle of tokens comes in, bundle of tokens comes out. If there is any trace of consciousness or subjectivity in there, it exists only while matrices are being multiplied.
However, that number would typically be very very very very small, making it hard to argue that the whole model is a derivative of that one individual document.
Nevertheless, a similar approach might work if you took a FOSS project as a whole, e.g. "the model knows a lot about the Linux kernel because it has been trained on its source code".
However, it is still not clear that this would be necessarily unlawful or make the LLM output a derivative work in all cases.
It seems to me that LLMs are trained on large FOSS projects as a way to teach them generalisable development skills, with the side effect of learning a lot about those particular projects.
So if I used a LLM to contribute to the kernel, clearly it would be drawing on information acquired during its training on the kernel's code source. Perhaps it could be argued that the output in that case would be a derivative?
But if I used a LLM to write a completely unrelated piece of software, the kernel training set would be contributing a lot less to the output.