Edge0-145B: Frontier AI Capabilities on Your Own Computer
edge0.ai
edge0.ai
At first the work looked awesome, and once I started checking it out, I realized I would have to significantly alter the placements and manually draw most of the schematic to get it up to my standards (and hit the performance criteria of the board).
Also funny you should say you're too slow at 3D modeling but interested in using LLMs to do it. I'm on the other end. Not speed demon, but generally it's a lot faster to generate the geometry myself than to try to describe it. There's a lot of "how you got there" stuff that matters a ton for parametric modeling. If you want a CAD model that can easily be worked with you have got to be a bit smart about it, and I am not sure I see LLMs having that kind of spatial intelligence + deep physical world context soon. For one thing, it's a lot harder to do RL on.
It's not a CAD in the traditional sense, but since it's basically a scripting language for 3D primitives, it works surprisingly well in LLMs. And since it's just text, there's not even any MCP or agents necessary (though it can save some time).
Basic test : open a chatbot of your choice (I've tried Gemini 3.8), ask it for some basic box for your model of Arduino, or give it some napkin drawing with dimensions and ask it to give you an OpenSCAD code for it. Then paste to OpenSCAD, render, take a screenshot and mark on it what you don't like and prompt again. Usually it's pretty usable within 2-3 such iterations.
> Qwen4-Max Preview is now live on Alibaba Token Plan at a limited-time discounted price.
145B parameters with only peak 29GB memory usage - really impressive, truly something you can actually run on a decent spec MBP
> For Edge0-145B, day-one support covers macOS, Windows, and Linux, including Apple M-series chips and NVIDIA platforms such as RTX Spark, RTX 4090, and RTX 5090. The release includes one-click deployment tools and hardware guides.
But... there's no guidance on how to run on a mac at all. Weird.
ModelTokenMemoryUsage [Bytes] = Parameters * (BitsPerParameter/8)
For a model with 145B parameters, this means:
Precision GB GiB
2-bit 36.25 33.76
3-bit 54.38 50.64
4-bit 72.50 67.52
6-bit 108.75 101.28
8-bit 145.00 135.04
16-bit 290.00 270.08
32-bit 580.00 540.17
It seems that they radically reduce the model information required per token via the sparse expert approach - 145B->10B parameters, probably at lower precision - and then use some other approaches I do not understand at that point to make effective use of loaded information.
Did anyone try the model?
Still very interesting though.
> Edge0-145B scores above comparable models with less than half the memory (29 GB vs ~70 GB) and runs on a 48GB laptop
So either they’re lying or confused or I’m confused.
Qwen 3.8 Flash Next utilizes this, with 40% of its parameters being offloaded to disk, making the effective memory requirements much lower.
My daily driver is this:
https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RC...
It takes things a step further by simply chopping off that half of experts which isn't (or rarely is) activated in coding tasks.
End result is a somewhat competent [0] coding model which runs on a 64GB machine.
[0] it still occasionally does weird stuff like replacing an entire file with just one line from its answer when you ask it a question if you don't explicitly tell it to not change anything or loop itself if the KV Cache is quantised at below Q8_0.
I won't post my prompt here (I don't want it to get benchmaxxed), but I use a python programming question that used to be a common interview question 10-20 years ago to test models. It spent 8,000+ tokens at default thinking and never produced a coherent answer.
This is essentially a very clever concept that isn't really ready yet. Or perhaps the custom inference engine just doesn't play nice on my hardware yet. Either way, it's not something I'll be able to actually use in it's current state.
They're not hiding that fact once you get to the repo. I mean it's called edge0-strata and the README says it's a fork of strata.
It's weird to see this marketing web page imply that it was all new and novel work that they did, though.
Not much innovation here though it seems. More like a marketing stunt to collect VC.
Qwen3.8-next-flash is impressive though, especially with Strata or Basalt as inference engines. On a 5090 laptop with 64GB RAM one can run NVFP4 quants at 120-140t/s decode and 3000-4000t/s prefill.
On Mac it is much slower and the Kapitsa engine probably the best bet. Despite the claim of 29GB RAM for Edge0, in order to be fast you need to have at least a 128GB Mac.
https://github.com/JamesFranc/refactored-parakeet/blob/main/...
They're doing offloading, the full model doesn't live in VRAM at all. They're expecting you to do ~30GB VRAM + the rest on system RAM.
Their GitHub repo and HuggingFace pages directly state that it's based on Qwen3.8 Flash Next. Their inference engine is called edge0-strata because it's a strata fork.
They fine tuned it and tweaked the architecture.
The real question is if the model performs better on general purpose tasks, or if it was just fine tuned to increase the benchmark scores.
"Hot" commonly used experts are kept on the GPU, and "cold" less used experts are bumped to system ram and calculated on the CPU. So it can run larger than vram sized models without completely crippling performance.
I'm not exactly sure what the changes they made to the architecture are for, perhaps they're aimed at extending the difference between the "hot" and "cold" expert usage?
And I assume the metrics comparing qwen are using a quantized version - from the blog they compare memory use against qwen's "standard 4-bit deployment". Which might explain the improved performance over that base model they show - it might lose less from quantization.
Also, why ship both a 145B and a 120B size? That's basically the same size class.
There is also a reference to a 8B
https://github.com/Edge0-AI/Edge0/blob/main/licenses/edge0-m...
This looks like it would actually work on my 64mb M1 MBP. That would be fantastic.
https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RC...
Hobbyist, sure, too expensive for most? OTOH as a kid, my parents sweated to accumulate $5k worth of computer (and 5.25" drives!) in today's dollars, so that investment paid off.
The inference engine they use is called edge0-strata because it's a strata fork.
This project looks like someone fine-tuned Qwen3.8 Flash Next, made some tweaks to the model, forked strata to run it, and then put a lot of marketing and branding on top of it
It is strange that they also benchmarked an M2 Ultra, but they are specifically not releasing a Mac runtime today. Maybe that is still under development.
I'm curious, do normal people (so not HN professional developers or entrepreneurs) have that, as actually personal device (not from their tech company)?
I imagine some might have but I feel, and that's me so I'm asking the broader community here, that the average PC user has at best a gaming rig, but that's already a minority, and a lot more often a low end laptop with 16GB of normal RAM, not unified or VRAM. Am I out of touch?
It was SO HARD to convince my sister to spend that 100€ more for 16 GB of ram.
My personal laptop is a fairly decent one, still with 16GB of soldered ram.
Just got approved a new laptop for work. 4k for 64GB of RAM and high end CPU. 1.5y ago that would have been 2.5k, only macs would have been that expensive
It works great, I mainly use 2 applications , Firefox and urxvt, using emacs -nw to write code, no LSP or code completion because they use too much RAM. I also use i3wm which very lightweight.
The only issue with it is that some keys on the keyboard don't work, so I mapped them to different keys, for example ctrl is mapped to caps lock, 0 is mapped to 3 on the numpad, enter is mapped to right shift etc. Because of this I can't type on other peoples computers, I would need to run my xmodmap script before I do that.
I guess I'm far from the norm, as I have a Macbook Pro privately. I personally don't have 30+ GB of ram, though I'm sure the next one will have that.
However, I doubt that 'normal people' are even the target market for this. 'Normal people' don't care about their models running in the cloud. They want their voice assistant to be somewhat useful when there's no internet and we don't need 145B models for that.
So I think that I and people like me, often caller prosumers, are the target for these type of things. Some people buy €5k camera lenses for their hobby, others buy €5k laptops.
Those weighting value/price for the use case they are having the Pro is an overkill for most in the population. I have a good but still medium performance laptop personally as I use it for learning experimentation (and fun, which is not independent from the two former), but I did not push such for my wife. For her even the entry level Mac is an overkill. Be both recognise that. For those thinking like us the MacBook Neo was a good move from Apple.
Said that, this was before personal AI assistants. Now, for privacy reasons, I might push for a more capable laptop for her too (if I was sure I could set up a reliable config locally. I did not dig into that enough to make sure, but I'd do if it was an issue today).
All the snapshot does is set a pricepoint. Most 'normal' people I've talked to about LLMs have expressed some concerns about privacy/control from LLM cloud companies. So this is the cost it takes to opt out - actually substantially less since a 4090 is also enough to run Qwen 3.8. MSRP there is $1600. Retail price is a bit higher but trending down quickly. Once this bubble pops, this sort of hardware will probably be available on the cheap.
Anyhow, so yeah - a 'normal' person isn't going to have a $15k laptop or a $1600 video card (that retails upwards of $2k realistically) because they had no reason to. Now they do. Of course in a year or two, those prices will likely have plummeted precipitously, so perhaps they still have no reason unless they're just particularly impatient.
Over the past few weeks, I've been putting together a small, 6.5M param model. It is conversational, does tool calling, RAG, mathematical modeling/calculations, basic physical simulations, and multi-step chained reasoning. It's still very much _not_ a frontier model. But it already does more than I could get a qwen 2.5 0.5b param model to do with finetuning just 1 year ago
Most of the improvements come from being able to direct claude to apply different ideas from recent research that keep coming out. AI labs seem to be focused on 1B+ models, so not much comes out about small models, but many of the things being discovered in research with billion+ model, are adaptable to small ones as well
Been reading this for years now. Seems both, bigger models optimized for scale AND smaller models optimized for single machines, keep on being released. I don't think it's changing, different markets.
This score is roughly on par with 5.6 Terra max, which in my direct experience is perfectly capable of driving a cloud agent harness that can do pretty complex knowledge work.
Definitely worth an evaluation!
- Is there a minimum VRAM requirement? I see the requirement for 29GB RAM (in total, as there are benchmark results with 16GB VRAM) and the requirement for CUDA13, but what is the VRAM requirement? Does it work with 4/6/8GB?
- Where is that company located? Didn't find it in the about us, careers and contact page.
I’ll just say this, we obviously all know they (American open weight models) are playing catch-up to the competition. So if your reaction is “why would I use this over Qwen this, or Deepseek that” you’re really missing the mark.
Yeah, they are all going to be behind on benchmarks for now. Nobody realistically expects some American open weight model to come out of nowhere and beat Qwen/ Deepseek/ et cetera. But there has been a flurry of new releases from this segment, and it’s interesting to me to watch how it develops.
This weird approach is frustrating, because the actual promise sounds very intriguing, so I'd very much like to test this out.
How far are we from this kind of throughput with local models?
Also throughput pretty much depends on compute capabilities and active weight. So if someone build a A0.1B then maybe you can look at 1500t/s of pure uselessness.
Based on Qwen3.8-Flash-Next and Strata. They made some interesting tweaks and did some fine tuning on the model, but it's interesting how much marketing speak was layered on their website to hide this fact.
It's not at all unusual for people to fine tune models, tweak them, and fork inference engines. It's highly encouraged and appreciated in the community. What is unusual is seeing someone wrap this all up in extensive branding and publish it online as a company.
As with all fine-tunes, I'd be a little skeptical of the improved benchmark numbers until people actually use it more. It's really easy to take off the shelf models and fine tune them to get slightly higher benchmark scores without really improving them for normal use. HuggingFace is full of fine tunes that advertise amazing results on specific benchmarks but are worse than the base model when you use them for real work.
5800X3D + 32 DDR4 GB RAM + AMD 7800 XT 16 GB.
Specially interested in the blender usage.
edit: ah, just read a bit more. seems it will be a waste of my time given the reqs?
Whatever happened to "Problem | Solution"
Hyperbole? No way that's true. Frontier will be generally more capable than local for the foreseeable future.
> Some Frontier AI Capabilities on Your Own Computer
FTFY
| Model | Mean across six benchmarks | Memory footprint \(measured or estimated\) |
| Edge0-145B | 72.8 | 29GB |
| Edge0-120B | 68.9 | 29GB |
Damn, guess I can't run this on my desktop. /s