mips_avatar

I recently had to turn off posthog on my app, it was collecting so much information that wasn't needed that it was making my app unusably slow. I'm sure i'm missing some knob, but the fact that after an hour long claude code session i couldn't figure out how to fix it means posthog has gotten too fiddly.

Yeah the problem is what's considered an adapted database. If it was strict it would mean apps like Alltrails (which is 90% openstreetmaps data) would need to list their trailmaps as open databases, but they don't.

If it hadn't been for Starlink I wouldn't have been able to work remotely from my parents house during covid.

I made a mistake, there is a $5k config with high memory bandwidth. The Max chip has two tiers (I incorrectly thought the tiers were based on memory capacity), you need the higher tier Max GPU upgrade (+$300) to get the 614 GB/s memory bandwidth but you don't need to upgrade the RAM to 128 GB to get the full memory bandwidth. So to get the 614 GB/s you need to upgrade to the max chip + upgraded GPU, but you can spec it at only 48gb if you want. So the total for an m5 max with 614 GB/s memory bandwidth is $4999-$9999 depending on config.

Still 3x lower memory Bandwidth than a dual 3090 setup which you can build for $3k with parts from facebook marketplace and run in your basement.

I benchmarked mine for a deep research workload I was running. Concurrency 1 is the speed you'd get if you're chatting with one agent,

2x3090 (has an nvlink bridge though it didn't seem to matter hugely for inference)

Qwen 3.6 27b int4: Concurrency 1: 68 tok/s output Concurrency 32: 363 tok/s output Prompt processing speed: 1520 tok/s

Qwen 3.6 35ba3b int4: Concurrency 1: 150 tok/s output Concurrency 32: 1083 tok/s output Prompt processing speed: 4324 tok/s

Macbook Pro m3 36gb RAM: Qwen 3.6 27b int4: Concurrency 1: 18 tok/s output didn't measure the other metrics and it was a slightly different benchmark.

You need the 128gb ram config to get the 614 GB/s bandwidth (which is $6999), you could skip out on upgrading the storage to save money but at that point I think most people upgrade the storage too at which point it's $8-10k + tax.

The cool thing about the 3090s is the RAM bandwidth. Token generation is mostly bottlenecked on memory bandwidth. Dual 3090s have 1.87 TB/s memory bandwidth (0.936 TB/s each), vs the M5 Macbook pro with only 0.3 TB/s (max chip has up to 0.63 TB/s but it's a $10k machine at that config).

This translates to qwen 27b actually working fast enough for useful work on dual 3090s and being painfully slow on Macbook Pros. Also if you're running a big model on a macbook pro the UI gets laggy and the keyboard gets hot. Much better to run dual 3090s in your basement and connect to them from your Macbook.

Here's my build! https://jonready.com/blog/posts/local-llm-rig.html

I love it because the watercooled 3090s are completely silent even under load. Facebook marketplace is definitely the move for a lot of the parts unfortunately, since you ideally would have higher end parts that are 2-3 years old.

I think the sweet spot right now is 2x 3090s and a pcie 4 motherboard with 64-128 gb of ddr4 ram, you can build this right now for $3k and it runs qwen 27b/35b stupid fast at int4.

I think it's going to be like DMCA, like hard to convict you for having the files but distributing them might be illegal

The problem is they've convinced mainstream people that a model that can find a bug in Microsoft Windows is a bigger problem than Microsoft not caring about fixing it.

I think it's more that they're abandoning simpler AI tasks to chinese models. Qwen 35b and deepseek flash are better than gp5 mini on my tasks and way cheaper.

This is what OpenAI/Anthropic want, it's better marketing than they can pay for -- and it creates a precedent for permanently banning the next generation of open weights models

OpenAI/Anthropic are begging to be restricted because it's great marketing and it creates a precedent to permanently ban open weights models. The problem is nobody in government believes in/cares about commodity pricing of truly open AI and how much it could help the world economy and prosperity.

Companies like Microsoft have been asleep at the wheel in terms of security for decades and now there's a model that can identify where they've been careless. That's not a "nuclear bomb level threat" or whatever Anthropic wants to call it, it's reckless carelessness by the existing companies.

My GPUs at home produce 60°C water. I just wish I had a good use for them.

It wasn't a slight provocation that injured me, it was a car hitting me

As someone who bike commutes my highest risk of dying this year is from biking. I've been hit twice biking in Seattle.

Everyone I know who bike commutes also owns a car and pays taxes on it.

I think we're about to be overwhelmed with good software that isn't great.

Well Anthropic would love some regulatory capture.

Makes sense, I keep hoping some startup will be able to crack the open licensing without limiting their business problem. Would love it if we could do a bit better than the rugpulling dynamics that are kind of common now without just giving AWS a buffet of startups to hurt.

This isn't about how to build an AI-native startup, it's how to use Anthropic tools to automate 2019 style app building. An AI native startup would have AI infused into the product, and Anthropic doesn't want anyone but them to sell AI.

Cool that you licensed it with GPL, what was your thinking on the license?

They’re not safety guardrails they’re anthropic doesn’t like anyone who isn’t anthropic working on AI rails

Anthropic specifically said that those notifications are temporary and fable5 will only pretend to help you if it’s ml classifier gets tripped

Anthropic is trying to hide bad behavior by being vague, it's important to not be vague when calling it out.

From the model card: "the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning" aka they will take your ML research code and inject bugs into it until it breaks using a LORA (or some other form of PEFT)

Yeah we can probably figure out how to run it on xiaomi gpus

They've said that they'll stop notifying developers when this gets triggered, instead they'll load in basically like a LORA that's designed to inject bugs into your code.

I guess it’s better they’re open about doing bad things. But now it’s a problem that they think this sets a precedent. They are one step away from feeling justified in using claude code running on a deepseek engineers laptop to hack deepseek to destroy their training system.