846 karma · joined October 30, 2021
I would like to redo the setup myself.
The "internal" memo: https://www.businessinsider.com/microsoft-is-offering-buyout...
octet => byte, https://fr.wiktionary.org/wiki/Gio
You may need a AMD EPYC processor and associated motherboard, don't forget a good PSU at least 1200W (300W processor + 400W by card to support spikes of consumption). On some setup (~15k€) you can use 4 CG to go up to 128G of VRAM.
I'm not using a EPYC processor for now, so I get 170tps on decode but very slow prefill, the bottleneck is the PCIe exchange between the cards, I need to setup P2P (it means what you think it means, for GCs).
My setup supports full context (2*18 bytes so 256K), I didn't setup YaRN to get 1M of context (2*20 bytes), but it's an option with 128GiB of RAM, the visual component is extracted from RAM so a little slower. I'm not using multiple agents but some people does, it's god like with 4 graphics cards, on 2 only, it's relatively quick but limited to single user and up to 3 to 4 concurrents requests only.
It's not frontier, but look AA of QFN, it's good enough. DeepSeek V4.1 can run on my setup, but it's slow. On dual B70, there is some PoCs to make it usable (>60tps). It improves every days.
If you need more information, look for local setup users on the Web. There is a lot of people sharing information about benchmarks and setups. L80 is my reference for now.
Edit: I translated the unity.
A lot more with Haiku equivalents (Qwen3.8 27B) or gemini. It's not frontier, but not that far away.
I can't load this page.
Otherwise, assuming there are Zionists everywhere is borderline antisemitic.
But the hack has already been disclosed and it's the responsability of the manufacturer to ensure that the fridge keep ever up to date or offline.
As with the USA Defense hack, it could have been prevented.
The hack: https://blog.claroty.com/team82/research/freeze-the-controll...
The budget they present on 6 of October is absolutely not social or even support a welfare state. The only thing "socialist-like" is retirement at 62yo, only for people starting to work at 20yo.
The vote at the national assembly state that they are right or far-right oriented.
- https://www.lemonde.fr/idees/article/2026/10/08/au-rn-un-ser... - https://datan.fr/groupes/legislature-17/rn
The main one is RN, François Asselineau is maybe the more "lefty compatible" far-right candidate, and he is clearly "right" labelled, not a lefty.
but are fine with Mullvad having the right to say that.
Copyright (c) 2026 maan
So... Either alooshdenny stole the commits, or it's an alias for maan.
What matter the most and isn't told by token price is the latency. You expect a voice LLM to respond very quick. If it takes 5s to response to a simple "Hello, what the weather today?", them not much people will use it.
So, the cost of a setup to run Qwen-Flash-Next at +40tks is around $3000. Too much for most people.
With only a RTX 4090, you will reach 30tps (with DDR5...), not +40tks, and it's about the limit to be usable. Oh ! I forget Apple device too, it's a good option to run this model I guess, but still slow.
Yet, as you said, it's still a wip implementation, it may improve soon (MTP support is about to be merged in llama.cpp soon).
Yes, I'm not a native English speaker, and I'm definitely not an AI. ;)
I can't say the same thing because I just don't notice typos (mine or others).j I just assume my English is bad.
Isn't the definition of "being human" is "not to be perfect"? In French, it kinda is. We say "He/she humain after all" to mean that someone made a mistake.
Yes... I'm not english native. And definitively not an AI.
Disclarer: I'm unsing Vulkan on an AMD GC.
in/out ($) Gemini : 1.5 / 9.0 | Qwen 3.8: 0.15 / 0.47
That is a massive cost reduction.
Refs: https://www.alibabacloud.com/help/en/model-studio/model-pric... https://runware.ai/gemini-omni
The last one was: Qwen3-Omni-30B-A3B https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct
And maybe Qwen4 won't be released, they only release Qwen3.8 27B (and a mostly unusable 125B). There are definitively slowing down open weight release.
So, you may actually have very good performance with local model. Just not yet on *every* device. So the Mozilla strategy here feel very reasonable. A Cloud provider specialised in local models, to be able to switch once local models will be quick enough on most devices.