cjbprime

Does the bug we're talking about not count as an example for some reason?

They did a specific-hardware kernel optimization for the Pro 6000 (available on AWS as g7e).

Cool! I went to a LAN party in a retired nuclear bunker in 2000. I remember someone had one of the first TFT screens. It was muddy, had zero visibility unless you were directly head on, and what felt like around 3fps. They were trying to play Counter-Strike. (Everyone else had a CRT and laughed at it.)

Looks like its only for 16/32-bit ops, which are almost irrelevant to LLM inference, but are relevant to training.

Is your argument that it would be really bad if it was true, therefore it's not going to be true? It would be really bad if I got cancer, but I don't respond to that by refusing to think rationally about it.

It has particularly good observability (the ability to see the full content of every prompt and response and tool call) compared to other harnesses, that's the main thing that stood out to me.

> I’m more and more convinced that all of AI engineering is Neijuan (内卷, meaning curl inwards). In China it describes a system that demands ever more effort and competition without improving output

I don't know what to say, except that articles exactly like this one have been showing up constantly for the last three years, and literally all of them were obviously outdated and irrelevant within about a month.

It could be literally automated. And I'm talking about while they were still minutes away on approach, not during final landing.

And yet I am! Care to elaborate?

I mean, complete ethical nihilism is one way to try to avoid losing this particular argument, but I don't think you'll get many people to agree with you on it.

Maybe ATC could be expected to revoke landing clearance on extremely obviously unstable approaches? I wonder why that didn't happen.

They made a series of at least ten (and that's kind of charitable) this-must-never-happen mistakes that likely rise to the level of extraordinary criminal negligence, including ignoring checklists, alarms, configuring the plane for landing in general, and basically everything related to safety.

You don't have to extend sympathy, just as you don't have to extend sympathy to drunk drivers who kill people.

> LLMs can't read binary directly without a disassembler.

They totally can. They're remarkably competent at disassembly.

The tokens spent to find the vulnerabilities will cost money, and the attackers will usually find themselves more financially incentivized to spend money on finding the vulnerabilities than the defenders.

If it was a mechanical keyboard with that case instead of an entire computer they could charge twice as much for it.

Waaait what? That's both crazy and crazy useful.

Not sure about that, V100s are all selling for less than $1k despite being 24-32GB VRAM around 1TB/s memory bandwidth, which is the same price point that 4090s and 5090s are commanding $4k-$5k for. Hardly hotcakes.

> strong recommendation: ignore that default. Run Qwen 3.8 27B on low or even no reasoning levels at first. It’s a great model, but wow that default setting is a bad place to start.

Has anyone tried asking the model to choose and emit the most appropriate reasoning level for each prompt, as the first part of answering it?

> General knowledge: I usually ask 2 questions many small models get wrong: summarize Operation Trojan Horse by John Keel, give publication year. Summarize the Ariel school incident of 1994. Qwen3.8 got the first question right, along correct publication year, gave glorious detail on Keel's theory, but got the second wrong. It thought that school was located in the USA. Ah well.

I'm not sure that this means anything. You're asking a ~27GB file to have losslessly compressed the entire training set (which apparently is a large chunk of the entire internet). That's not possible. Whether it happened to encode these particularly obscure facts losslessly or vaguely isn't really telling you anything about how good a model it is.

Why not? The web uses TLS, how's it different security-wise compared to a package download?

Does anyone know how to get this working with Claude Code via llama-server? I'm getting a jinja template error about the system prompt not being the first message.

Hm, I have a 4090 as well, and:

$ build/bin/llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spec-default --cache-type-k-draft q4_0 --cache-type-v-draft q4_0 --threads 24 --jinja --reasoning on -fit off

0.02.993.689 E ggml_backend_cuda_buffer_type_alloc_buffer: allocating 911.53 MiB on device 0: cudaMalloc failed: out of memory

Update: Oh, it works after I stop Xorg. But nvidia-smi only showed Xorg using 200M out of the 24G, so why would a 911M alloc fail?

Does it not use Metal, on macOS? Would it be faster if it did?

It's a very conservative warning. The application does not perform writes, so the application doesn't actually wear your SSD at all. The rest is just application-independent general hygiene.

You still have a core misunderstanding. Only one layer of weights is required in memory at a time. A forward pass can be over-simplified as a matrix multiplication of each layer, one at a time.

There is no swapping of working RAM. We're just talking about loading the weights read-only data into RAM on-demand for each layer. It is only as slow as your storage interface.

Both sentences are likely wrong. It's a written lifespan (technically an "erased" one), not a read lifespan. The weights are only needed from disk read-only. And Mac NVMe interfaces are surprisingly fast.

Edit: Oh, I think you maybe thought I meant swapping working RAM off disk? I didn't. I meant swapping weights off disk into RAM on-demand.

I've got access to a 192GB RAM Mac Studio, which is below the stated minimum RAM. Can swapping off fast disk be used to make it work out, especially since it's MoE?

ChatGPT reconstructing idiomatic Python source code from Python bytecode was definitely up there. That is not something humans have written a great deal about online. It requires simulating the Python VM.

I remember also having a massive wtf reaction to realizing that original ChatGPT was pretty good at decoding long random/unique base64 strings.

> If it detected any harmonics it would be too high.

I think it's not that simple. A tuner is "hearing" the fundamental and all of the harmonic overtones combined. It has to guess at which frequency is the fundamental, even if the overtones are actually stronger than it amplitude-wise, and it does that by looking at the nature of the repeating overtone pattern and extrapolating back to the fundamental.

I think you can end up an octave too low (half the actual frequency) if the waveform repeats in a way that implies a different overtone repetition pattern, for example if there's an every-other-cycle artifact to the waveform.

Inference (not training) is bottlenecked by memory access speed, not compute. Having special hardware wouldn't make it faster unless you somehow found a faster memory controller than the GPU has.