scosman

The top of my ~/.claude/CLAUDE.md:

> Don't turn a one-off or area-specific comment into a durable memory without my explicit confirmation. You have a history of over-indexing on one-offs, and those memories end up getting cited to override well-tuned skills.

Kimi K2.5 >> Haiku. Gemma 4 32b might fit the bill.

Look at Qwen for that level of intelligence.

short answer: they mostly aren't

A few people are running highly quantized models with limited context windows. It's still impressive, but not the benchmark level intelligence. Very few people could afford a rig for reasonable local performance at a reasonable quant, at full context size.

The antirez example is 2.6bit quant, 32k context, and few tokens per second... on a ~$7000 MacBook M5 (new RAM pricing).

I dunno. I know a lot of software engineering experts. AI isn't always right, but neither are the people, and it's getting better and better.

Software is one domain where it excels because of structured training data and simulation environments, so I'm well aware it's better here than other areas.

Still there's somewhere balanced between saying every time it's "insufficient or incomplete or outright misleading" and "just trust AI". AI's a useful source of information/reasoning/research, but know you need to validate it's answers for important decisions.

This was copying many screens, composing layouts and many strings. not a "general look and feel" or one off utility string. Others found parts of the code copied (exact variable names). It would take a court to decide, but I think it's pretty clear cut copyright violation. They gave their agent had access to the EE code, and seemingly asked it to produce a copy. It's not a 1:1 copy, but there was a lot of copying. No clean-room attempt was even made.

If I take the first chapter of Harry Potter and write a new ending, it's still copyright violation. They took many parts.

They copied strings, and visual design. Both of which are work, both of which are covered by copyright.

Short segments of popular works sure. Many UI pages with identical layouts and copy, essentially zero chance. The agent had access to the original code at inference time.

"ghostlygourd.com" is a S+ tier domain. Would click

It's not economic to run them locally. It's amazing for privacy, and fun hobby. But you're either looking at super slow CPU builds with $10k in RAM, $90k worth of GPUs, or a really quantized model that doesn't compare in quality.

I might build one for fun, but it's not going to change the economics alone. Still exciting it's possible.

anyone manage to get a point?

Yeah totally agree! My pet peeve feature list is also better for society than donating $500M to charity!

Top right: link to the Lore Repo on Github

My 8yo son plays with this any chance he gets. We've started building watches together as a result (simple "drop in movement mods", but aspirations of building our own movement). The author really made something special.

I think the point is: if every app chooses the best model for their use case, my phone is hosed (disk, memory swap, memory). A good-enough default might be better for the user than the each app having the best possible.

ahh yes. I'm using Whisper v3 turbo via WhisperKit as well. Will play with parakeet

I think that's what they are trying to avoid. If you need on-device intelligence, their pitch was "The model the device already has is best", and if you need something more specific an adapter (aka, a fine-tune/lora) is best.

They were wrong when their on-device model was way behind. They still might be right in the long term.

While multiple app I use might need Gemma 4 E4B, I use dozens of apps and app devs can choose from hundreds of models. A shared cache might reduce size a little when there's overlap, but the core problem still exists. If each app chooses a model disk and memory-swapping explode.

Its probably be better for device manufacturers to bake in a default. I'm not proposing they limit you from using others, but one shared default might be best developer/user experience for 99% of apps.

- Being warm in memory is the single biggest perf speedup you can get, and a default is much more likely to be warm.

- "Best model" is usually "best model for this device" given both RAM and compute. A developer can't test every device but Apple can/will.

- Each model needs to be optimized for the hardware (what's running on ANE, what's running on Metal, what's running on CPU). The default gets optimized.

- If you need custom model, a Lora is probably best (30MB, benefits from all of the above)

You could say the default should be swappable, but that's more a linux ideal than an Apple one so I doubt we ever see that. Plus there are real downsides: intentional or not, prompts end up optimized to the model they are developed for, so swapping the default system model would degrade every app.

I had the same experience so started building my own. All problems are solvable, just working on the polish.

- crash recovery: part one is use ADTS aac (even if process crashes, audio is saved up until it does). Part two is isolating the transcription/summaries in separate XPC services.

- disk space: AAC 64kbps mono soles it. Could use Opus for further reduction but both are small.

- speaker bleed: macOS voice isolation processing solves this. It’s a nightmare to get setup, but works great once done.

- library: using argmax SDK - by a bunch of ex-Apple on device AI folks.

It it wasn’t for CoreAudio, I’d say it was easy to make. Argmax, Whisper, and llama.cpp - wrapped in the right architecture, mostly just work.

I’m having fun nerding out on the details like custom vocabulary (get the names of the people in here meeting right), inferring speaker names from transcript, calendar integration, nice UI, etc.

Those transcription times are fast fast. What model/library do you use?

They actively use the name https://www.war.gov

This is likely correct, sorry for the bad info. Was working from memory.

Free server-size model access for apps with <2M downloads, getting the same privacy guarantees. Hopefully they scale this up to all apps in time (I assume hardware/cost constrained, but larger devs would pay).

https://developer.apple.com/private-cloud-compute/

> just a privacy-polished frontend for Gemini

I'd use this.

I'd rather have strong privacy guarantees, but this is still good.

Cerebras is trialing Kimi K2.6 at 3000t/s (invite only). I'm excited for when the fast hardware gets more mainstream for frontier models. Models designed for speed on Nvidia are nice addition that could bridge the gap.

Google's QAT claims to need 6.7 GB RAM, vs Unsloth's dynamic quants at 8GB. Would love to see some benchmarks. Both amazing for size.

The image on the Nvidia announcement tells the story: 6 identical laptops from various manufacturers running the same SoC. The Microsoft one is just one of many.

https://www.nvidia.com/en-us/products/rtx-spark/

They also do a good job working over the little differences between APIs. Tool calling sometimes breaks on major providers, and OR will patch it before the provider does. Libraries like LiteLLM do this too, but OR is faster.

until GPT 5.4, Claude always had a decent edge in benchmarks (any date before then). The gap was huge during the Sonnet 3.x vs GPT 4.x eta.

Benchmarking 1 or a few samples isn't ever going to yield anything but noise. The actual benchmarks use thousands of tasks.

GPT 5.5 genuinely was back on top for a while there, but if you look at the past 2 years, being on Claude was better than being on OpenAI most of the time. If you're going to pick a tool and not switch constantly it was the right choice. Not to mention their tooling has always been ahead, and that gets ecosystem benefits.

Are they close and interchangeable today? Sure. But Sonnet was genuinely way better than anything OpenAI offered for a long time -- the valuation reflects that, not any given moment in time.

not your fault, the article uses that language!