tarruda

If they baked in a function to freeze USDT, why not also add a function revert a transaction flagged as part of a hack?

I stand corrected then. Thanks!

I've always preferred GPT over Claude, but TBH the releases after GPT 5.6 Sol plus the surprise of cutting my usage in half is making me want to jump to anthropic and give Opus 5.5 a shot.

I wish they would give the notice one month in advance. Last week my $200 was renewed, and if tomorrow my usage cuts in half (if I understood correctly) it feels as I got partially ripped.

What is the state of virtio-gpu these days? I have a system76 "pang14" (Ryzen 7 7840U laptop), is that a valid option for me to have a VM with 3d acceleration? I'm fine with having some overhead if it is better than the current software GPU.

> The thing is that as soon as you seriously give a chance to Bcachefs you see how good it is.

I'm pretty sure Bcachefs is amazing and better than Btrfs. I also think Zfs is amazing and better than Btrfs. Even so, I still use Btrfs because I know it is guaranteed to always be present on any Linux without any effort on my part.

> That might be the best thing happened to the project since now development can happen at its own pace without the clicky bait influencers.

A better approach might have been to just paused mainline merging instead of forcing being kicked out?

Eg "Hey Linus, Bcachefs is still in early development and I need to merge changes in a pace that is not compatible with Linux development process. So I'm going to pause for a while now and once it reaches maintenance status I will focus on submitting patches in a healthy pace that you can digest".

Except for the fact that the developer has sabotaged the project into being removed from mainline?

At this size (~400 million parameters), does it become viable running directly on CPU?

The exception is if you use CoW filesystems.

Btrfs has a workaround: it can create a cow-disabled subvolume for the swapfile.

AFAIK you need a separate partition for Zfs.

Awesome!

Darn, for a moment I thought it would be possible to run in 128G T.T

200B of those 552B is PLE, which works more like a database that is read for each token, thus can be offloaded to a fast SSD.

> It also includes additional 196B Engram memory which you can put on an SSD. I think

You can put Qwen 3.8 Flash Next engram on SSD, but prompt processing takes a good hit. On my mac studio, I get 300 pp and 33 tg with SSD offload, versus 550/40 with everything in RAM.

I will be very happy if 300 pp is achievable with this model though.

I tried dwarfstar when llama.cpp DSV4 support was still very weak, and while it worked, I didn't see anything that would make me want to stick with it vs llama.cpp. llama.cpp is simply better with its awesome built-in webui, router and server APIs and certainly support much more models and quantizations than dwarfstar.

Since then, I started maintaining my own vibe coded dsv4 branch with metal optimizations, so I actually get much better metal performance on my llama.cpp branch than on dwarfstar (plus all the extra llama.cpp features). Here it is in case you want to give it a shot: https://github.com/tarruda/llama.cpp/tree/qwen4exp-dsv4-opti...

Don't use my Vision-Exp GGUF though. As I said I built those GGUFs before llama.cpp supported, and they can't be loaded on current master (require my own branch).

I already have new GGUFs but haven't uploaded yet. If you want Vision-Exp, maybe use bartowski or unsloth's GGUFs.

Side note:

As an alternative to deepseek v4, you might want to give it a shot at qwen 3.8 flash next. I have IQ4_NL GGUFs that can be loaded fully into 128G, or Q5_K GGUFs that can offload the PLE to disk (use --load-mode none --lazy-mode on for that): https://huggingface.co/tarruda/Qwen3.8-Flash-Next-GGUF.

llama.cpp master is still somewhat bad in Qwen 3.8 next performance, but I was able to achieve 40tps tg and 600 tps pp on my private branch.

I would rather have just 3 levels: low, medium and high.

IQ3_XXS (~3.2 BPW). For me this is an option because my Mac studio is only used for serving LLMs, so I can afford to dedicate most of its RAM to this. I can run with 256k context and only uses ~117G, with the remaining (up to 125G which I can allocate to VRAM) being used for prompt caching and context checkpoints.

I'm making my own quants, though the Vision-Exp version is outdated and won't work on llama.cpp master branch (I built it before llama added support):

- https://huggingface.co/tarruda/DeepSeek-V4-Flash-0731-GGUF

- https://huggingface.co/tarruda/DeepSeek-V4-Flash-Vision-Exp-...

For the Vision-exp version, I also ran perplexity + KLD against the original MXFP4. Seems quite OK: https://huggingface.co/tarruda/DeepSeek-V4-Flash-Vision-Exp-...

Hopefully it will be open weights and have the same architecture and size as the current v4 flash vision, which is probably the best LLM that can be run on 128G devices.

I use my Mac as a headless server to run LLMs, and never even start a desktop session so it always stays at pre-login state without any applications running. Even so, I never saw it using less than 2G in htop when logged via SSH.

I'm certain the video is sped up. Still possible to script computer use, and I think this is what happens. One of the suggests this: https://x.com/DanielSMatthews/status/2096311244504572177

Could be hallucination, but I gave this to an LLM and this is what it suggested:

"

The workflow shown in the video—processing an image and then controlling a computer interface to draw it—is a combination of two well-established fields: Computer Vision and UI Automation.

You do not necessarily need a Large Language Model to perform the underlying image processing; standard algorithms can do this deterministically.

Step A: Image Processing (The "Brain")

You can write a script (using Python libraries like OpenCV or Pillow) to process the reference photo:

    - Edge Detection: Use filters (like Canny or Sobel) to find the "high spatial frequencies" (outlines).

    - Color Quantization: Use clustering algorithms (like K-Means) in the HSL space to group millions of pixels into a small palette of distinct colors.

    - Vectorization: Convert these processed shapes into a set of coordinates (SVG paths) that represent exactly where the mouse needs to move.

Step B: UI Automation (The "Hand")

Once the image is converted into a set of instructions (coordinates and color codes), you can use automation tools to physically control the computer and draw on Canva.

    - Browser Automation: Developers have already created projects that use Selenium (a web automation tool) combined with edge detection algorithms to draw images onto HTML canvases.

    - The script reads the pixel data, calculates the mouse coordinates, and executes the "click-and-drag" actions in the browser.

    - Computer Use APIs: In the case of GPT-6 Astra, the model uses a "Computer Use" interface. It effectively takes the processed image data (or generates it internally) and outputs high-level commands (e.g., "Move mouse to X,Y," "Click," "Select Hex Color #FF5733"), which the system then executes on the screen.

"

Seems plausible and easier to believe. Also, feels like a "magic trick" designed to fool the user into believing that the agent is drawing interactively by using its vision, since it could just have written a python script that takes the input image, and produces the exact same result without automating the screen.

Interesting, thanks for sharing!

A scripting API makes the problem much more approachable, but what about those videos where Astra is drawing people from a photo? Here's one using canva: https://x.com/iam_zachi/status/2095992132620136677

Is that also using scripting to batch updates? It does look as if the mouse is moving.

Amazing project.

My only blocker for installing Asahi on my Mac Studio (M1 ultra) is that llama.cpp performance is very bad when compared to using the Metal backend on the same hardware.

I haven't tried any of these demos, but I'm not surprised they stop impressing once you go deep.

What got be mostly impressed were the demos of Astra doing computer use. At my job I do some RPA and can appreciate how challenging it can be. Yet they make it look extremely easy to operate a tool like Blender at super human speeds.

X.com is full of unverified posts, but here's a few:

- https://x.com/MengTo/status/2096213835460084184?s=20

- https://x.com/birdabo/status/2096101381925114327?s=20

Seems really impressive at first glance, but there's no guarantee that they are real

> even need to understand that it's "bypassing" because it thinks it's just iterating towards its goal.

Could they have added a "no internet access" goal constraint?

Even the behavior of agents searching for sandbox bypasses must have been in the training data, or at the very least, "suggested" in some way.

To be this whole thing feels like a marketing play by OpenAI.

Also proud of having spent $2.5k on a used 128G M1 Ultra back in September 2024.

Despite being outdated in terms of compute, it stills let me run very good recent models locally, with Deepseek V4 Flash 0731 being the greatest one right now, and hopefully Qwen 3.8 Flash will also fit well when it is released tomorrow!

The M1 ultra definitely leaves to be desired in terms of its token speeds, but I think 20 tps generation and ~200 tps prompt processing (which is what I get with DSv4 flash), is already enough to do a lot of serious work when you combine with the decent prompt caching provided by llama.cpp.

> not that Apple prices weren’t insane before the ram/ssd shortages

Funny that after the price started to increase last year (I think October/November), there was a window of a few months where Apple prices stayed the same as they were before, thus making apple prices actually good when compared to the rest of the market.

It was a unique opportunity to have acquired a 512G M3 ultra for $10k.

Can you share the source for the parameter count (125B A6B)? I didn't see it anywhere in the page.

> I’m struggling to understand how that was causing such a large difference in the output.

It is incremental, the more a pattern appears in the context, the more likely it was to continue appearing in future turns.

So the model was likely trained to end a reasoning trace with a single linefeed and a `</think>`. It was also likely trained that two consecutive linefeeds sometimes produce an "Actually..." sequence.

So you can think of it as:

- the first time the reasoning trace was parsed, the two trailing linefeeds followed by </think> were added to the context by the template.

- next time the model was finishing a reasoning block, it added an extra linefeed instead of just closing directly with </think>. This slightly increases the chance that the next token will begin an "actually" sequence instead of closing.

- If it caused an actually, that was added to the context, further increasing the chance of a self correction at the end of the thinking block. The more self correction paragraphs are added, the higher the chance that following turns will have more.

- Eventually it can result in a state where it enters that loop forever (or at least for a very long time).

> Is the “autoparser” vulnerable to injections somehow?

The autoparser was (and still is) incorrectly parsing a trailing linefeed as part of a reasoning block. The were two ways to fix this, both of which must be implemented for the fix to be complete IMO:

- fix the autoparser definition to ensure remove surrounding whitespace is not returned as part of the text blocks

- trim leading/trailing whitespace in the encoding phase, so it fixes bugs or even "injections" where the client deliberately adds the whitespace to trigger problems.

For this specific issue, the maintainer later fixed by trimming the extra linefeed before passing to the template.

> How do you distinguish between user text, model text, and metadata, or is there ambiguity in the parsing?

That is model specific. Ultimately, a token stream is being produced and parsed by the inference engine, and each model uses different tokens/formats. The goal of the autoparser engine was to simplify the creation of parsers for new models by inferring the delimiter tokens from the chat template.

The llama.cpp API server returns pre-parsed data, so clients don't need to do any parsing to know what is a thinking block, a text block or a tool call.