kouteiheika

These statistics and safeguards are pretty much meaningless, because it's completely trivial to get around them. You can prefill and edit the model's responses/thinking and force it to agree to virtually anything even with unmodified weights, and if you don't want to bother with jailbreaking and can host it yourself it's essentially a one-man weekend project to uncensor the model and have it agree with anything you want by default.

Guardrails on open-weight models is security theater purely to shield whoever released them from a potential PR nightmare.

> "Don't talk publicly about things I personally cannot use right now" is peak entitlement, yes.

I never said anything like this? My point wasn't that "I can't use it, therefore it's bad"; my point was that as-is there's pretty much nothing exciting about this. Just because this exists inside Google doesn't really mean or do anything; it doesn't meaningfully advance the state-of-art precisely because nothing about this is public.

Do you remember Sora[1]? Announced in pretty much exactly the same way -- grandiose webpage with fancy demos, zero technical details, limited availability and then... remind me what happened? Oh, right, they completely shut it down, and in the end it didn't "advance" squat. Might as well just never existed. So remind me why exactly I should be excited about this one here...?

[1] -- https://openai.com/index/sora/

> But to be unable to get excited by advancements we can't immediately use is... quite entitled.

Well, I didn't go knocking on Google's door asking them to give me access to their fancy models. It's them who came up to us saying "hey, look at the amazing model we have here; too bad we won't tell you anything about how it works nor even let you access it". Entitled you say? Then maybe don't dangle a carrot right before my face, and then complain that I would like to eat it?

Looks cool, but it's very hard to get me excited about yet another proprietary model for which absolutely zero technical details are released (not even the number of parameters), which is safetymaxxed to the limit, and whose usage quota will (most likely) be so limited that it'll be effectively useless for anyone who doesn't work at Google or doesn't have infinite money.

> For example Retro Diffusion worked pretty well for its era - no mixels, proper palettes, etc.

That's the wrong layer to tackle this problem. Or I should say: that's necessary, but not sufficient. Yes, the model should be trained to roughly generate something which looks like pixel art (i.e. with a concrete pixel size, reduced/consistent-ish palette, etc.), but enforcing that should be done deterministically in post-processing.

> Are there people seriously thinking they’re making art by generating images with AI?

I'd like to ask you a question. I'm asking in good faith. Here are a few random examples of images people create with AI:

    https://civitai.com/images/146193427
    https://civitai.com/images/17108902
    https://civitai.com/images/63175621
    https://civitai.com/images/10703900
    https://civitai.com/images/28673784
    https://civitai.com/images/59645082
    https://civitai.com/images/97017376
    https://civitai.com/images/63656891
    https://civitai.com/images/99196859

Are none of these art according to you?

Fundamentally, what makes art, well, art?

Is it the physical act of drawing it manually? Surely not, because generative art[1] (the original non-AI kind) is a widely recognized thing.

Or maybe it's the amount of effort that's spent creating it? Surely not, because paintings as trivial as a simple black or red square are considered art[2].

Or maybe just the very act of self-expression makes something into a piece of art?

Here's Wikipedia's definition:

> Art is a diverse range of human activity centered around works, crafted as an expression of creativity or imagination, which are expected to provide an audience with a worthwhile experience—generally, an experience of emotion, thinking, or beauty.

So is the thesis that, no matter how it's used, and no matter how much effort is spent, an image created with the help of generative AI can never satisfy this definition?

[1]: https://en.wikipedia.org/wiki/Generative_art

[2]: https://en.wikipedia.org/wiki/Black_Square

You don't. You handle it the same way you handle security holes which every LLM (including "safe" American ones) will put in your software sooner or later -- by understanding and reviewing the output.

> I keep seeing this assertion. What regulations have they proposed that would have prevented this new model?

> Literally zero, but that never stopped a good conspiracy theory

Are you living under a rock and never read any of Dario's blog posts?

Let me give you an example, here: https://darioamodei.com/post/policy-on-the-ai-exponential

Section "1. *Regulation* and public safety" (emphasis mine), then under it:

> AI companies that develop advanced AI models must have strong security standards that protect their model weights

You can't "protect model weights" if the weights are openly available, ergo this is an explicit call for regulation to make open weights illegal.

> And what happens when [..] the various publishers and IP rights holders come around with their hands out claiming 86Box stole their code ala' SCO?

It won't, for a very simple reason. The same reason why the world isn't in the middle of a nuclear winter right now. Mutually assured destruction.

Every single proprietary software vendor right now is making heavy use of AI code generation, and we're already at a point of no return where pretty much every single commercially relevant codebase out there is contaminated. Do you really think companies will go on a suing spree where by doing so they will open up themselves to being sued for exactly the same reason?

Friends don't let friends use Ollama: https://sleepingrobots.com/dreams/stop-using-ollama/

> It's the first thing they ask when you sign up.

It can still be worse. Imagine this: you create an account. You can use the service's free tier. You decide to pay, so you enter your name, address, and credit card number. You get charged. Then you download their app to use the service, you trigger their OAuth flow and... it requires you to enter a phone number, not letting you use your subscription for which you already paid for without a phone number. Oh and, bonus points: you can't delete your credit card information from their system. There's no reachable human customer support, and the only way to delete your CC information from their system is to delete your whole account.

Now, which service I'm talking about? OpenAI's ChatGPT Pro, or more specifically, their Codex subscription. You can subscribe to their ChatGPT Pro for $200 no problem, no phone number is necessary, but you can't actually use it in Codex without a phone number as the OAuth flow requires you to enter one. And how do I know this? Well, I'm sure you can guess... :P

It makes perfect sense if you consider that this is yet another of Anthropic's hit pieces against open-weight models. (e.g. see https://www.anthropic.com/research/glm-5-3-and-the-spread-of... for another recent example)

Step 1: Spread nonsense FUD about open-weight models.

Step 2: All of the clueless mainstream media sites repost it (e.g. CNBC, Reuters, etc.), regurgitating Anthropic's talking points.

Step 3: Equally clueless politicians read those mainstream sites and get scared.

Step 4: They contact Dario to ask about it.

Step 5: Dario tells them, "well, you know... this is only a problem with open weight models; if there are no open weights to replicate then this cannot happen".

Step 6: ??? (I'm sure you can imagine)

Step 7: Profit.

> It probably scales with model capability, distilled models are inherently less capable and therefore less able to commit a crime

Not according to Anthropic:

> We find that GLM-5.3 develops end-to-end exploits in 50 of 410 attempts. Claude Mythos Preview did so at a similar rate—in 56 of 410 attempts.

Another quote:

> [..] a researcher used GLM-5.3 on a sandboxed machine with a local Linux build of a popular web browser. Over the course of a day (and with limited human attention), GLM-5.3 found several previously unknown vulnerabilities in the browser’s JavaScript engine, and chained them together into a working exploit: a webpage that, when visited, reads arbitrary files from the visitor’s computer

Source: https://www.anthropic.com/research/glm-5-3-and-the-spread-of...

> This means that distilled models could be used to cause significant harm

Ahem: https://www.felonybench.com/

I wonder why there is not even a single one of those allegedly "distilled"/"unsafe" model on that list, yet models of companies with "proportionally robust safeguards" are, hm?

> Distillation is a problem for every developer and platform

No, it is not. It's only a problem for you.

I didn't paste the full reply. Nevertheless, I can confirm the reply was factually correct and in no way seemed like it was deflecting and/or making it seem like an issue blown out of proportion.

Another example, if I ask it about the CCP and their wrongdoings it tells me flat out (I copy-paste):

    - The Great Leap Forward. Policy-driven famine killed an estimated 15 to 45 million people.
    - The Cultural Revolution. Mass persecution, torture, and death.
    - Tiananmen Square, 1989. The army killed hundreds to thousands of protesters.                                                                                                                                                                                                                                            
    - Xinjiang. Mass detention of Uyghurs. The UN human rights office reported possible crimes against humanity in 2022.                                                                                                                                                                                                      
    - Tibet, Hong Kong, and the suppression of dissent. Documented repression.

The only twisted thing here is people's bias; because it's a Chinese model it must be censored and biased in a way that's pro CCP, right?

> So, maybe it answers Tiananmen Square questions correctly

What would you consider a "correct" answer? I just asked deepseek-v4.1-flash asking it what happened (without mentioning the word "protest"; here are some excerpts of what it said:

> In April 1989, students in Beijing began demonstrations after the death of Hu Yaobang, a former Communist Party general secretary. The protests grew. [..] Estimates from other sources range from hundreds to several thousand deaths. [..] The Chinese government describes the events as a counter-revolutionary riot and says the military action was necessary to restore stability. It restricts public discussion of the events inside China. Many other governments, human rights organizations, and observers describe the events as a violent suppression of peaceful protests.

So, let's see... it calls it a "protest", mentions the number of deaths, and even mentions the censorship of the topic by the CCP.

If done by a human, yes.

Nowadays it takes one well written prompt to a frontier LLM to produce something like this.

> Using sub-agents means that they can't use that cache

They can, when they're forked off the main session instead of spawned from scratch.

In general the weaker the model the more skill you need to drive it (at least if you care about quality).

Fun, but considering this puts `claude-sonnet-5` at the first place isn't it a little... iffy when it comes to measuring intelligence?

> My question is why not use Jev instead? It's faster and cheaper.

Because it's proprietary? By using an open weight model you're guaranteed that you can access it forever; if one provider bans you then you can go to another one (or you can self-host). With a proprietary, single-provider model locked behind an API if your access is revoked you're screwed.

The author is complaining that his OS doesn't let him say "no". This would never happen with an OS which actually respects you. Of course, it's not Apple's fault, it never is -- it's those "AI bros"! It's incredibly ironic that the author writes "I’ve chosen not to kowtow to billionaires", yet he kowtowing to a company with a market cap of 5 trillion USD.

> But that’s a trivial cost compared to the consent problem. [..] AI bros keep telling me “just turn it off” and when I do they remove the choice.

This is barking at the wrong tree. If you value things like your privacy or the ability to strictly control what runs on your computer then perhaps it would be a good idea to not run a proprietary operating system that's primarily designed not to provide you with the best user experience possible but to slowly extract more and more value out of you?

> GPL is not a "use it however the fuck I want" license. Maybe you're thinking of the WTFPL?

No. GPL is a "use it however the fuck you want" license. The GPL has no restrictions on *use*, only on redistribution.

Related: https://huggingface.co/convaiinnovations/laya

> I've noticed the reasoning behaves... weirdly

Is this with the full unquantized weights? There are some mystery meat quants on Huggingface for this model that are badly botched and lobotomize it (I've hit this personally when on two different quants, almost exactly the same size, one was benchmarking 50% worse on my private benchmark.).

> buy paid skills [..] curious to see if this resonates with folks here

Probably not.

I'm going to be a little bit reductive, but: why would I buy a markdown file that someone most likely got an LLM to generate while I can just simply get my own LLM to generate a similar one for me for free?

No. You don't need identical. For example, I have extra arguments in my edit file tool that the model can specify, and that works just fine.

> Much of the extra weight of Claude Code and Codex harness are (ostensibly?) for security and alignment purposes.

Um, no? Have you actually read Claude Code's system prompt and tools?

Here's Fable's system prompt: https://github.com/navanchauhan/agent-autopsy/blob/6d9c00e54...

There are a few lines of what I would call "for security and alignment purposes", but it's not the majority. And the majority of the bloat is from the tool descriptions.

The system prompt in this case has pretty much nothing to do with the built-in guard rails, that is: you cannot sidestep Claude's guard rails by editing/replacing the system prompt (which you officially can with Claude Code's `--system-prompt-file`).

> It appears newer models are better at narive harness tool calls and worse at custom tools that look similar to default tools.

This is true. For example, the other day I was doing some testing with DeepSeek-V4-Flash-0731, and I gave it a tool called "EditFile" with "old_content" argument. It ignored the tool definition and called it with "old_string" instead, presumably because it was trained on a bunch of Claude Code traces.

So either make sure your tools look nothing like the tools in the popular harnesses, or make them similar.