I was already rolling around the idea of a 128GB M5 Max MBP. Now this!
A 4-bit MLX quant with 128k window should fit perfectly, in the 50-70 tok/s range.
783 karma · joined May 15, 2014
A 4-bit MLX quant with 128k window should fit perfectly, in the 50-70 tok/s range.
https://www.reddit.com/r/StableDiffusion/comments/1vciy35/lt...
Perhaps refined ComfyUI workflows will squeeze more quality out of it, but it's definitely not in the realm of Seedance, and Lightricks is training LTX 2.5/"LTX-Next".
I swear no matter if it was busy or not, it seemed like everyone was assigned to the same one or two elevators. The wait was always so long too, I used the stairs most of the time if just going <6 flights.
Someone besides RCCL needs to study that algorithm.
Got 48 tok/s decode at 1.9 GB RSS (2.4 GB peak), faster than the 24 GB M5 Pro mentioned in the benchmarks. The ~2.0 GB/s SSD number quoted for M4 is the base chip. This M4 Max does ~7 GB/s.
Page cache seems to be why it beats the M5 Pro. With 64 GB the whole 12 GB packed_experts set stays resident, and iostat shows only ~1.6 GB per run actually reaching disk, against the ~79 GB that 98 fully cold tokens would need.
I then tested with DaVinci Resolve open and under load (playback): 42.6 tok/s. Also held 38 GB of incompressible memory to squeeze the page cache: 41.8. At 48 GB it ranged 32 to 41.5. Degrades gradually rather than a cliff. It's a beautiful thing.
You're probably burning tokens. A markdown file with a summary, structure of your project, rules, and even goals prevents Claude from hunting for and reading files over and over. Each new session (or after compact) would be starting from scratch. I take it a step further and create a HANDOFF.md file between compacting that gives the next session full context of what was done and how to continue.
Also:
"Treat CLAUDE.md as the place you write down what you’d otherwise re-explain. Add to it when:
Claude makes the same mistake a second time
A code review catches something Claude should have known about this codebase
You type the same correction or clarification into chat that you typed last session"
"How many pelican riding bicycle SVGs were there before this test existed? What if the training data is being polluted with all these wonky results..."
May I ask how much did the training cost you?
Would you trust living in a high rise designed by AI?
Designing a system that survives production is the job.
Just before asking for approval to run, it said one thing it wanted to "flag before running" was "Rate-limit and auth testing against prod will generate some 4xx noise in Railway logs and could trip the form rate limiter — harmless, but saying it now."
Ok fine, I said go for it, and it says:
"Running it. Quick recon first (prod URLs + the prior-findings baseline), then I'll fan out the audit tracks with adversarial verification."
Immediately after, I got the Fable warning about how it can't continue because of safety concerns, switching to Opus. In the end, Opus did a good job thanks to whatever Fable suggested doing. Things were fixed that Opus missed in a security/performance audit just the week prior. But what surprised me is that it used 55 agents. Burned 80% of my 5-hour window in 15 minutes (5x Max plan). I've never had Opus do that before on these audits.
> Today’s agreement will quickly expand our available capacity, delivering meaningful compute in the next three months and nearly 1GW in total before the end of the year.
They need a bunch of compute, now.