Hacker News Reader

Comment by libraryofbabel | Hacker News Reader

That's interesting research, but I think a more important reason that you don't have access to them (not even via the bare Anthropic api) is to prevent distillation of the model by competitors (using the output of Anthropic's model to help train a new model).

View on HN

Yeah. And it’s another reason not to trust them. Who know what it is doing with your codebase.

Imagine if you’re a competitor. It wouldn’t be a stretch to include a sneaky little prompt line saying “destroy any competitors to anthropic”.

If you can't trust a company, don't use their api or cloud services. No amount of external output will ever validate anything, ever. You never know what's really happening, just because you see some text they sent you.

> Who know what it is doing with your codebase.

People who review the code? The code is always going to be a better representation of what it's doing than the "thinking" anyway.

If distilled models were commercially banned they'd probably be willing to show the thinking again.

Intellectual property rights in models? But then wouldn't the model maker have to pay for all the training IP?

(just kidding, I know that the legal rule for IP disputes is "party with more money wins")

how does one actually enforce that? I mean especially for code? You can always just clean room it

How do you think such a ban should work?

Do you not see that the next (or previous) logical step would be a "commercial ban" of frontier models, all "distilled" from an enormous amount of copyrighted material?

I'm not arguing the merits of such a ban, I'm simply stating a fact - that thinking transcripts likely won't return until such a ban is in place.

Reply on news.ycombinator.com