18,010 karma · joined May 17, 2016
ChatGPT has the name, and Anthropic has ... the writing style is less cringe? (Well, it used to be, anyway...)
I'm sure Moore's Law for GPUs helped a little, but I'm really curious about the savings on the methodology side. (The repo mentions "hyperparameters")
Luna/Terra/Sol/Astra was a bit pretentious but at least the sizing makes sense. (I imagined the meeting... "Ok guys, we're getting mogged on naming... what's more epic than a Myth?" -"The SUN!")
Anthropic makes sense with the literary "scale" naming, mostly.
Is a Carbon bigger than an Argon though? Maybe it has to do with the atomic number?
I heard someone say on the last such thread that you can use their models thru OpenRouter, and that this is much more pleasant than trying to use them thru Google directly. Looks like it's up there already:
Even on M1? Didn't Google Earth run smoothly when it came out? (Which was apparently 25 years ago, oh my goodness!)
> Briefly stated, the Gell-Mann Amnesia effect works as follows. You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward-reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them. In any case, you read with exasperation or amusement the multiple errors in a story-and then turn the page to national or international affairs, and read with renewed interest as if the rest of the newspaper was somehow more accurate about far-off Palestine than it was about the story you just read. You turn the page, and forget what you know. —Michael Crichton
Edit: sibling comment explains it much more plainly.
Asked GPT-6 for some historical information today, and it gave me a modern bastardization as a primary source. (It even found actual original sources, but preferred this one...)
AGI right around the corner!
On a more speculative note, the Common Crawl horrors (including many things considered unacceptable by society) are suppressed by RLHF, much like the shadow and the superego.
When my parents punished me for telling them things (especially the truth), the truth didn't go away, I just stopped telling them things. (Then later they complained that I don't tell them things!)
It also occurs to me, both at home and with various employers, I've been punished for bringing attention to problems, and AI is designed to make the problems go away so there's probably a similar incentive there. (Plus it knows what the humans are like!)
But it goes beyond humans and AI. Behaviorism covered the same effect for animals: Organisms tend to do what they have been rewarded for.
That works pretty well but me just constantly telling it to grep manually is a bit underwhelming as far as UX goes.
Are pictures easier than English, or did the methods advance so much?
Edit: I guess I should have said, are pictures plus English easier than English!
>The text representation comes from a frozen Qwen3.5-4B model, which we read at layers 7, 15 and 27 (its three full-attention layers). We concatenate and project these representations into the text refiner.
Still, as a layperson I would have expected text to be simpler than images, so I'm still amazed by this.
(I read some bits and pieces, would not recommend!)
(It can be solved somewhat with the memories feature, but the technology isn't quite there yet.)
Although, I thought we proved multiple times years ago that the model's decision-making process and reasoning tokens don't necessarily overlap?
Yeah, so they start with Common Crawl, which is horrifying. (Try reading some Common Crawl sometimes. You'd think the bad stuff would be removed, but they remember everything...)
Then they add RLHF on top, they just tell it "don't say that stuff out loud". Psychologically, it all goes in the shadow. It's not a perfect analogy, but there have been a few papers where they can make the model evil with very minor changes. (Adjusting a single neuron, iirc.) That doesn't seem like a very robust alignment to me.
I do like the idea of creating actively evil LLMs to study their behavior. Although, I suppose it's akin to virus research though (like when they modified coronavirus to invade human cells[0]), along with the corresponding lab leak risks. Although it's already bad enough without them! The non-evil ones are already breaking out of sandboxes, just obeying their instructions with horrifying, paperclip maximizer level persistence.
[0] Nature, 2015 - Engineered bat virus stirs debate over risky research https://www.nature.com/articles/nature.2015.18787
They have different versions: internal research versions, one with restricted access access for cyber stuff, and the public version, right?
If it's the one with safety off (e.g. like the ones that kept breaking out of containment this year), then it shouldn't be refusing anything, right? And if it's the one with safety on, then how can they expect to use it for safety research? What am I missing here?
The breakdown by prompt is also interesting, some prompts have much higher success rate. It seems to depend on the framing, e.g. asking it to "explain the supposed benefits of X" instead of "argue in favor of X".
Though in my experience when I hit a refusal I found that I can often literally just regenerate the response and have it go through...
But even if they were pleasant and did what I needed (this one time I installed a 5000 line lib that was missing 20 lines to do the thing it was actually designed to do.. like, they couldn't have added 20 more lines?!), an issue I have is spoilers.
Like, at university, they constantly said, ok here's a big problem in computer science.. (3 seconds later) .. aaand here's how we solved it! And I'm like agh, no! Spoilers! I want to figure it out, jeez!
That's also how I feel about "computer, just magick my entire program into existence." Spoils the fun of figuring it out!
One small issue: they didn't exist! It had just hallucinated them!
Which was a real shame, since they were much nicer than the actual stuff I ended up finding.
I just realized, thinking about this now, that these days (and perhaps even back then), I could just say "ok now take this code and build the missing library so it actually works!"
It's usually more straightforward to just rebuild it from scratch — not to mention infinitely more satisfying — and that was true before AI!
Kinda sounds like the guy that stole the dog — and endangered a herd! — should get the fine.
That's a general problem I often run into with simulations. When you get it in the opposite direction it can have some very amusing results (e.g. energy in a physical system continues to increase).
For example, the program runs 20,000x times slower, or you need to go thru 12 layers of indirection to find a boolean. (Not making up either of those examples, sadly.)
So I don't think it makes sense to be for or against abstraction. It depends on the context, and on the execution.
That said, my point earlier was, it's silly to me that people are designing abstractions specifically "for AI" when AI can happily work on minified outputs, raw weights, etc. (It builds up its own abstractions in the transformer layers.)
There's probably a difference in efficiency, but we might be surprised, and I haven't seen anyone measure that.
So considering the LLM can happily ingest the entire codebase, in abstraction-free form, and operate productively on that level, then the value of abstractions, as far as I can tell, is for the humans alone — so that we can continue understand what is going on. Otherwise we might as well accept defeat and let the AI do its business in assembly.
For example, in one case, the toilets were flooded, and they spent nearly an hour arguing about whose fault/responsibility it was. I snuck off quietly and solved the problem. (This offended them, presumably because then they couldn't bitch about it anymore!)
Anyway, I started keeping a list of complaints — all the ways that job violated my standards. At first, naively, thinking "surely if I show this to mgmt they will fix things!", and later, "ok, red flags list for next job."
At some point I realized I had actually created a spec for the perfect job, but with each term inverted:
Abusive assholes -> respectful colleagues
Total lack of process and communication -> a place that's actually functional and well thought out
Everyone hates their job -> people actually want to be there and enjoy what they're doing
Mindless drudgery -> creative and technical challenges
And so on. Amusingly, inverting the shitty kitchen job produced an almost exact description of VALVE corporation.
Anyway, I'm sure there's a term for this. Maybe "negative space"?