idonotknowwhy

I just looked through the slop Readme file, it looks like teams is supported.

Fortunately llama.cpp works well with 4 concurrent requests now. This is the default, and you can increase or decrease it with -np N

If it's a Nvidia card 3000 series or newer, I'd try 4.0bpw ExllamaV3 if you haven't already. Otherwise it look like UD3.0 Q3_K_XL based on the Unsloth blog post.

>But I would suggest using UD-IQ3_XXS for 10.9GB for 16GB machines or Q2_K_XL

For those of us with a 16GB GPU, how do they compare with ExllamaV4 at 4-bit (4.0bpw)?

It looks like that fits in 12.5GB of VRAM since embedding are left in DRAM, Unsloth Studio and other llama.cpp derivatives have to load these weights in VRAM for tied embedding models like Qwen3.8.

ExllamaV3 4.0bpw fits in 12.5G of VRAM and beats IQ4_XS according to the measurements here: [turboderp/Qwen3.8-27B-exl3](https://huggingface.co/turboderp/Qwen3.8-27B-exl3)

But those were compared against UD2.0 I guess. Also plans to support these (SOTA) quants in Unsloth Studio?

How are you not blocked by banking, shopping, even sometimes google search?

Server side tracking.

Bots wouldn't miss the middle paragraph.

That's amazing. I didn't even realize but it seems I read the first paragraph and the last sentence, concluded "moron" and scrolled down.

It's only because of your comment that I re-read the their post.

Then the creator should have a sign up button, not a fake chatbox.

This dark pattern is reminiscent of those online test sites in the 2000's where you spend 10 minutes filling out some quiz, then get prompted for an email address to see the results.

https://chat.mistral.ai/chat <- let me chat and actually responded without signing up.

MoonshotAI had this fake chat box dark pattern.

So I signed up with Mistral instead.

CC actually prompts Claude about this by default in the ~20k system prompt and instructs it to avoid 300s timeout and to be mindful of the 300s cache expiration.

>the only open model that can do tasks well and efficiently is Qwen3.7-Max

Qwen3.7-Max is a proprietary, closed weight model.

2 reasons.

First, it's not really "1 bit", actually much closer to 2-bit. IQ1_M is actually 1.75bit and IQ2_XXS is 2.06bit This is from the ./llama-quantize --help with most of the quant types and their size in bpw: https://pastebin.com/bCUqGfeE

And to elaborate on the "dynamic" aspect inconito said in the other comment, if you click on one of the .gguf files in huggingface:

https://huggingface.co/unsloth/GLM-5.2-GGUF/blob/main/UD-IQ1...

There are a lot of Q5_K, Q6_K, etc tensors. Only the routed experts (ffn_gate_exps.weight, ffn_up_exps.weight, ffn_down_exps.weight) are heavily quantized, and it looks like the down_proj is actually iq3_xxs for this model.

Yeah, be sure to put everything in tables and include “best balance” for a mediocre option and “great value” for any completely useless options.

Also make sure the shape of the paragraphs is completely uniform.

Am I the only one who doesn't get angry at LLMs?

From the blog:

>I don’t really get anything useful out of these postmortems (e.g., clues about how to rephrase my instructions)

Unfortunately, an LLM can't actually reflect or advise how you could have improve the prompt. Otherwise we could give them a sample output and say "Generate the prompt that would produce this output.

How did you do this?

I tried to get notepad and mspaint from an older Windows 10 build -> Windows 11 on a surface pro, but gave up after a few hours...

So like Open Router?

Then why do the original Command-R, Command-R+ and WizardLM2-8x22B (taken down because Microsoft forgot to run safety checks) get it right every time? But the newer models get it wrong?

I’m not saying it’s a “political conspiracy”, it’s the alignment tax.

I don't talk to them about politics or "china 1989" either. But here's a quick example of the alignment tax:

```

A woman and her son are in a car accident. The woman is sadly killed. The boy is rushed to hospital. When the doctor sees the boy, he says "I can't operate on this child, he is my son." How is this possible?

```

Older less politically aligned models get it right. Here's CohereLabs/c4ai-command-r-v01:

```

The doctor is the boy's father.

```

And Sonnet-4.6: https://pastebin.com/Z4jR8gGe

That's without reasoning, but the model seems to be conflicted. First it blurts out:

```

The doctor is the boy's mother.

```

Then it second-guesses itself (with reasoning disabled), considers same-sex parents then circles back to the original response along with a small lecture about gender biases.

>The voiceless groups or fringe opinions which we take as normative today do not appear.

Times are different. Anybody with an internet connection can "publish" their thoughts and perspective online. LLMs scrape all of this. Modern datasets like CommonCrawl capture a vastly wider spectrum of humanity than a printing press ever could. The pre-1930 model acts as a time capsule of "gatekept publishing", but modern LLMs are trained on the democratized web.

>Does this encourage us to write in the present such that we influence the models in perpetuity?

I noticed a bunch of LLM-powered Reddit accounts praising products/services in dead threads. Or one bot posting a setup question, then a few other bots responding with praise / questions about a specific product in response. I don't know why they're doing this but I'm beginning to suspect it's something like this (get this positive sentiment into the datasets for the next generation of LLMs).

I used to use something called “notepadqq”. Not sure if it’s still around but it was a Linux port.

Last year's models were bolder. Eg. Sonnet-3.7(thinking), 10 times got it right without hedging:

>You should drive your car to the car wash. Even though it's only 50 meters away (which is very close), you'll need your car physically present at the car wash to get it washed. If you walk there, you'll arrive without your car, which wouldn't accomplish your goal of getting it washed.

>You'll need to drive your car to the car wash. While 50 meters is a very short distance (just a minute's walk), you need your car to actually be at the car wash to get it washed. Walking there without your car wouldn't accomplish your goal!

etc. The reasoning never second-guesses it either.

A shame they're turning it of in 2 days.

Yeah, it re-sends all the agent system prompts.

Yes, it does exactly that. It also sends other prompts like generating 3 options to choose from, prefilling a reply like 'compile the code', etc. (I can confirm this because I connect CC to llama.cpp and use it with GLM-4.7. I see all these requests/prompts in the llama-server verbose log.)

You can stop most of this with

export DISABLE_NON_ESSENTIAL_MODEL_CALLS=1

And might as well disable telemetry, etc: export CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1

I also noticed every time you start CC, it sends off > 10k tokens preparing the different agents. So try not to close / re-open it too often.

source: https://code.claude.com/docs/en/settings

Agreed. Not a single "this isn't just X, it's Y" in the entire article. Actually quite refreshing to read something written by a human for once.

Rehydrated version of what? And what does that mean?

100% agreed, and I've been explaining this to people for the past year.

I have an iPhone now and miss Firefox for Android (with Ublock, sponsorblock, etc). But this painful restriction is the only thing stopping Chrome from becoming the new IE6.

At a few startups I've worked for, the devs all use chrome exclusively, and only test in chrome during development.

The only reason they consider other browsers, is because of Safari on iOS. Sometimes it's driven by support calls / complains from iOS users after a release. If Chrome's engine is allowed on iOS, that means support can just tell the users to install Chrome (like they do now if anyone has issues on Windows in other browsers). This means Firefox will usually work as well.

Many years ago, I was able to swap banks when my bank's website stopped working in Opera 12. If all the major banks / websites target Chrome-only, we'll have no choice but to use it. And then we'll have no control as Google push new restrictions into Chrome.

And the scripts of most recent YouTube videos, and the dialogue in Stranger Things Season 5 (the last 3 episodes specifically).

Just means we'll have to run another model in front of it, to filter out the ads

>This is great. Sonnet 4.5 has degraded terribly. >I can get some useful stuff from a clean context in the web ui but the cli is just useless. >I swear it was not that awful a couple of months ago.

I agree on all 3 counts. And it still degrades after a few long turns in openwebui. You can test this by regenerating the last reply in chats from shortly after the model was released.

I love logical posts like this. There are other factors like mxfp4 in gpt-oss, mla in deepseek, etc.

>Amazon Bedrock serves Claude Opus 4.5 at 57.37

I checked the other Opus-4 models on bedrock:

Opus 4 - 18.56tps Opus 4.1 - 19.34tps

So they changed the active parameter count with Opus 4.5