And, please, be mindful of the difference between arguing discriminatory policies (racism) and finding the facts about the world we live in (science). And it would be interesting to know what contributes more to groupthink, genetics or environment.
1,440 karma · joined September 24, 2015
And, please, be mindful of the difference between arguing discriminatory policies (racism) and finding the facts about the world we live in (science). And it would be interesting to know what contributes more to groupthink, genetics or environment.
[1] https://git.musl-libc.org/cgit/musl/commit/?id=b3c4cc121f70f...
A model has internalized and most likely generalized mechinterp research. Yes, it's expected and nothing insane. Just like internalizing and generalizing math research was expected and not insane. The results of the latter are... interesting though.
Isn't it a common practice to use cheaper models to do the grunt work?
You, probably, talk about something else.
Oh! Do you know how does the brain function? Heck, we don't know in detail how LLMs do their thing. The trivial observation that the brain in not a Turing machine is irrelevant if we are talking about functionality and not computational substrate.
But a Turing machine that implements functionality of a target system is the information-theoretical optimal compression of the data produced by the target system. So it stands to reason that LLMs that need to compress vast amounts of data approximate such a machine.
> C++ is Turing complete. Hell, HTML+CSS is Turing complete.
This is absolutely irrelevant to the question of the limits of LLM training. It's a common surface-level patter matching response, though.
> There's a very obvious and immediate one: availability and size of training data. And compute. And time.
Yeah, practical limitations. And practical results like seven hundred solved math problems show that you need, at least, to stop and think.
> What does quantum computation have to do with brains or LLMs?
If you have to ask that, it means that you haven't dug into all this mess deep enough. Basically, BQP computational complexity class is more powerful than P-class. If the brain employs such computations (although we don't know the appropriate quantum algorithms, so it's just wishful thinking mostly), the corresponding Turing machine is impractically large and slow.
> Says the person who just listed a lot of beliefs in marketing bullshit.
Nah. Learn more.
An LLM with CoT is Turing-complete, so there's no wall to its capabilities. There's no theorems that establish limits of reinforcement learning either. There's no evidence of quantum computations in the brain.
All you have is a strong belief.
The wallet is at $2,016.73
Peephole optimization that can't be made global due to combinatorial explosion?
For example, "Many Worlds, the Born Rule, and Self-Locating Uncertainty" notices that the rule doesn't mean anything, but it is rational to behave like it does. It's not a very satisfying position, I'd say.
No. They haven't been caught stealing. Tristan Buckmaster publicly expressed surprise that the OpenAI model has used the niche approach he pursued (with AI assistance). 10,000 agents exploring various approaches make it much less surprising.
In other words, the method of assigning CVEs hasn't changed, but the number of CVEs has risen sharply.
ASI (artificial superintelligence), theoretically, can exploit the ultimate analog nature of digital circuits (something like Plundervolt, but more intricate), but if we'd get to this stage when we need to defend against ASI, we'd have bigger problems.