electroglyph

ONNX has AVX512 CPU kernels too, and openvino uses ONNX internally (and ONNX supports openvino backend)

it's a pain in the ass to do properly.

what we really need it something like auto-round for ONNX

ONNX is my first suggestion to people looking for speed gains on CPU

heh, i came here to say basically the same thing

existing embedding models like alibaba's modernbert tune or one of the jina v5s would probably map query to category automatically. (i.e. store embeddings of each category and calculate cosine sim for each incoming query vs. categories and pick the closest)

also, you could stick a classifier head on a BERT model as another option.

heh, when Raymond Chen dunks on the MSVC team =)

it's probably a lie

deepseek 4 pro is insanely good for the price

that link actually recommends not doing it from UEFI and doing it via software

any divergence (even if the benchmark is better) from full precision is error

this is better than TFA

sometimes i wonder if the left hand knows what the right is doing. it looks like we arrested our own spy in this case: https://www.politico.com/news/2026/05/25/american-journalist...

absolutely. somebody online was wanting an LLM with Georgian language support, and that's exactly what i suggested: start digitizing Georgian text.

i'll upvote this each time it's submitted

you should be using dflash with that model, look it up

heretic maintainer: https://github.com/p-e-w/heretic

the fun bits are in another branch or PRs

p-e-w was just talking about this the other day in his Discord. seems doing the one neuron method is quite bad for KLD and that's why the newer techniques have stuck.

site has so little information there doesn't seem to be much to discuss

this looks awesome. i've been struggling with vector compression, and have been trying PCA + all sorts of rotations. looking forward to trying this out

nice writeup! looking forward to doing some more training as soon as i get some more data sorted. it'll be a custom arch, but i'll probably shoehorn it into unsloth for a speed boost.

you can train it, but not fully

that's in the ideal scenario where it's only seen a single copy of it tho

it was 1.3e-6 billion years ago!

i'm doing inference on a free mi300x instance from AMD right now. not sure if the software stack is just old or what, but here's what i've observed: stuck on an old version of vllm pre-Transformers 5 support. it lacks MoE support for qwen3 models. oss-120b is faaaar slower than it should be.

int8 quantization seems like it's almost supported, but not quite. speeds drop to a fraction of full precision speed and the server seems like it intermittently hangs. int4 quantization not supported. fp8 quantization not supported.

again, maybe AMD is just being lazy with what they've provided, but it's not a great look.

right now the fastest smart model i can run is full precision qwen3-32b. with 120 parallel requests (short context) i'm getting PP @ 4500 tokens/sec and TG @ 1300 tokens/sec

but should you drive or walk to the car wash?

i dunno, Opus is losing it's edge imo. i regularly use a mix of models, including Opus, glm 5.1, kimi 2.6, etc. and i find that all of them are pretty much equally good at "average" coding, but on difficult stuff they're nearly equally bad. i can't deny that Opus has an edge, but it's not a huge one.

> they also don't know what they don't know

they sort of do tho:

https://transformer-circuits.pub/2025/introspection/index.ht...

how about a unicode art tool?

https://electroglyph.github.io/atheriz_draw/