Don't ignore the possibility Claude was trained on non-public documentation or code, perhaps even illicitly acquired. That doesn't make Claude any less useful, just not necessarily for the reasons you think.
It worked flawlessly but lets be honest. It's not like Claude checked random numbers with data slice and observed what e-paper did
Datasheets are almost impossible to find and companies typically resell and hide original cheaper manufacturer here so they have no incentive to share that
- My experience isn't unique. Social media is flooded with stories of similar superhuman reverse engineering feats by frontier models. For example, someone has created a replacement OS for Rigol gear (oscilloscopes, etc) using AI [1]. Is all of this just in the training data?
- Watching Claude work, he seemed to be going through the motions of actually reverse engineering it. First he dumped the ROM, disassembled it, and identified undocumented opcodes. He created many test programs to probe the various memory areas. It all struck me as very meticulous.
Now none of that is incompatible with the idea that some of this was already in the training data. But I guess I want to push back on this urge to always come up with a reason for why the AI isn't actually as good as it seems. At this point I think we need to face the fact that human programmers are already being eclipsed. Yes, it terrifies me.
Do these wild "what if" even matter? I mean, what's your point? That it was trained in a way that it enabled this sort of task? That's like putting out a disclaimer that the water is wet.
Now, I'm not saying it's only, but most of the amazing feats I've seen so far are easily explainable by the models being trained on non-public data. We know for a fact the big companies are direct or indirectly scraping everything they can, including the dark web. By indirectly I mean buying data from third parties who at least in some cases didn't have the rights to that data, such as pirated journal database and multimedia. Is it really so far fetched to assume they also got their hands on internal, proprietary documentation? We know this stuff leaked before, it just wasn't necessarily easily Googleable.
> To the best of our knowledge, the agents in this pipeline discovered and used at least 10 novel Linux exploitation techniques.