Distillation attacks are covert, industrial-scale campaigns to extract the capabilities of one model and replicate them in another, without the authorization of the original model’s developers.
Replace a few key words and it sounds like reading all the worlds media and offering a service that summarises it.
Don't scrape the scrapers, that's not fair! Expecting Executive Order against 'distillation' in 3, 2, 1...
Seriously though, these clowns at Anthropic are not just hypocritical. They are clearly feeling threatened by open source AI. I almost couldn’t believe the new blog post today where they argue that dangerous open source models might soon be running on (checks notes) smart ovens.
The laws they are arguing for are not going to protect anything, they essentially are waging war on open models and since most open models that are 'pretty good' to 'good enough' hail from China they have chosen to play the China-fear card.
But they won't be able to stop anybody from doing this if they want to continue to operate Claude as a business going forward, and that is assuming that is what is actually happening. Chinese labs have been extremely open about their developments, they have shipped working models to the likes of me, which is more than I can say about Anthropic. On top of that they don't sermonize about how they are going to improve the world and never mind the accidental destruction, that's just collateral damage.
Probably stamping heavily on much of the nuance, there's a certain irony in any of the frontier models complaining about others stealing their work. The secret sauce for the frontier labs may be the reasoning layer, but the secret sauce of any piece of literature is the specific ordering of the words. I hope the parallel is obvious.
The frontier labs are spending record-breaking amounts of money on hardware (to the extent that they're locking out anyone else from the market, but that's beside this particular point, kinda) in order to create their service, and another company that is 'distilling' has not paid the same hardware toll. How much did it cost an author to create their literary work, which could be read and ingested for a miniscule percentage of the cost it took to create? For the benefit of the service provided by a (wannabe) multi-trillion dollar company.
If the frontier labs can prevent or slow down 'distillation' actions that customers are using their service for, then that's all well and good. Maybe they write an acceptable use policy that says it's not allowed (and they probably have). Good on them, it's their service. It's the way this is framed that I have a problem with. Referring to it as an "attack" and lines like this:
Second, illicitly distilled models present a national security risk.
It's bloviation. It's fear mongering. It's being one of the biggest, most powerful and influential companies in the world at the same time as playing the victim. Feel free to share the methods being used to prevent what is considered unacceptable use, but let's not pretend these are the only companies having to expend effort to protect trade secrets and company value. Let's also not pretend this post is all that much more than "lead-up to IPO" hype.
Today, movies make money from in person theater tickets and records make money from tours. I don’t know what the equivalent would be for AI (maybe IRL robots?) but it does not look promising.