Given that Anthropic has never released anything open weights I wouldn’t count on the fact that they view finetuning Gemma 4 as something allowable. I think they think nobody other than Anthropic should have AI
I'm really uncomfortable with these changes, like everything Anthropic's doing as "frontier research" today will be regular product engineering in a year.
It's bad that Anthropic can determine what this means. If you're building a modern app you're likely training your own embedding models and now anthropic can just silently sabotage your training pipelines?
I don't think we've bottomed out on what we can do with embedding models. They're these tiny models that absolutely rip on modern cpus with 8 bit int optimizations. Like in my app we can say pretty definitive things about hundreds of millions of places in the world on retrieval tasks on regular hardware.
Still somehow was "impossible" for the Seattle police to recover security camera footage of my bike being stolen under the light rail station security camera.
Microsoft is trying to sell things like extended servicing agreements. They purposefully make Windows worse so they can sell you solutions to fix it. They purposefully keep it insecure so you need their updates. It’s about taking the customers hostage.
I think if modern LLMs were invented in the mid 2010s it would have been promoted in more positive ways, but because everyone is afraid for their economic security saying scary things gets more of a response. I think it's kind of gross that it's a race to scare ordinary people and especially Dario Amodei should feel kind of ashamed of himself.
It's the same problem as fireworks, the only models supporting LORA are like year old dense models that perform horribly on most tasks. If you want to do anything close to relevant you still need to rent/own dedicated GPUs, which seems insane to me when vLLM fully support dynamic LORA loading.
Maxes out around 4K tok/s output. Each pair of 3090s has its own instance of the model with parallelism across the nvlink bridge. Though nvlink is only 2x over pcie5
So it's basically just openrouter with cloudflare argo networking? I feel like they could do some much more interesting stuff with their replicate acquisition. Application specific RL is getting so good but there's no good way to deploy these models in a scalable way. Even the providers like fireworks which claim to let you deploy LORAs in a scalable way can't do it. For now I literally have to host base load on my application on a rack of 3090s in my garage which seems silly but it saves me $1k a month.
Well it’s just more that I’ve noticed in the agents I’ve built that qwen doesn’t get reliable until around 27b so unless you want to rl small qwen I don’t think I would get much useful help out of it.