vikramkr

I would argue for anything security - if there is a difference between how it works in theory vs in practice - then the theory is flawed. Physics gets to have theory/practice distinctions, but engineering doesn't because the bridge can't fall down.

This doesn't have the vibes of something that's designed perfectly even in theory. This more has the vibes of "this was the least worst option"

Based on how quickly the CTF was won - yeah that seems very reasonable.

It was exploited within 2 hours of the server being fixed to be vulnerable it proved the exact opposite of shat you're saying

It was true. The author had messed up the server and it was not exploitable. It was hacked within a couple hours of the author fixing it (even with the server being down) but the author had also messed up and not put the actual Bitcoin wallet in it so it took an additional 5 hours for them to fix that. It was a mess but the vulnerability was very real

I mentioned this a couple times there but the unintended consequences of that challenge are wild. 2 hours after it the server was fixed there was a successful exploit but because it was originally misconfigured a huge number of people walk away thinking the vulnerability was not serious. At the least we now know a good way to convince large swarths of technical folks to not take new vulns seriously! Which seems not good

There are auditors and the security, plus since IPOs involve raising money from institutional investors with very deep pockets those folks are a more credible legal threat than retail

honestly the surprising thing at this point would be if america did actually pass a policy of any sort, good or bad, vs just relying on the president to yolo some executive orders now and then that change every 2 years. Plus at least for the next couple years nvidia likes open source models and the president likes the ceo of nvidia. There could be a falling out but nvidia has deep pockets to ... discourage ... a falling out.

It's worth remembering that the tariff stuff is not any sort of durable policy. It's one guy sort of doing whatever until 9 other guys a year later go 'hey wait' and then the first guy does whatever again anyway. The whole thing is just a recipe for whiplash. maybe congress remembers that they're supposed to do stuff in a few months and steps in. maybe they don't and then every 4 years we have wild swings in enforced based on which half of a percent of the voters in georgia had rainy weather where they lived in early november. More of a simultaneous "All of the above and also none of the above" deal until the dynamics of american government fundamentally shift

Tokens per second? That only really matters for things like chat and sessions where you are monitoring a session as it runs which is going to be an increasingly small percentage of usage. That's bottlenecked by human attention. Agents in the background, long running tasks, unattended tasks, all that stuff is what is even capable of driving significant usage because you can just increase consumption massively by parallelizing without being the bottleneck. I wouldn't be shocked if already the vast majority of token usage was from unattended sessions. TPS is not that important there

I think at least a few of the replies including mine are saying this is a pretty meaningless over hyped demonstration because the cnn was fine tuned on the same dataset it was tested on and the actual way the eval was structured and run is nonsensical. It's basically just slop research designed to create a fun attention grabbing headline (ironically one that trashes llms).

Presumably with the level of interest in it people have been checking what the theorem definitions are to make sure they align and I would expect Anthropic to be burning a huge number of tokens to rip into the proofs and figure out if any of them are exploiting bugs in the lean kernel etc. at a point it's as close to irrefutable as you can get

The crisis is from what it has already borne out (navier stokes + proof dump). Potential continuing improvements at anything approaching the rate of improvement they already are showing is layers of crisis on top of that.

This is not a compelling analysis.

First, the CNN is not 10 years old. The base is but it was fine tuned for this project on the same dataset the test images were derived from:

> The test: The lesions come from MILK10k, a public dataset released through the International Skin Imaging Collaboration [ISIC] in 2025

> The CNN is deliberately not state of the art. I used Inception v3 pretrained on everyday photographs and then fine-tuned all layers on 68,011 dermoscopic and clinical images from the ISIC Archive on a single H100 GPU

The test images themselves were filtered out at least, but that's not really enough. The CNN has a home field advantage because the test images are going to be in distribution with the training images.

Second, Looking at the code, for frontier models probably you should be giving them image tools and using them in an agent loop and not trying to one shot results (e.g. https://platform.claude.com/docs/en/build-with-claude/prompt...). It is also not meaningful to ask an LLM for a numeric confidence score.

Third, all 200 of the the test images were biopsied by the clinician originally. So when you ask the model "You are assisting a dermatologist" and "Decide whether this lesion can be safely discharged, meaning it needs no treatment, no biopsy and no follow-up" why would you penalize the model for making the same call the physician did even if it was benign in retrospect?

Are you evaluating the model on if it can replace the need for a biopsy itself (in which case the ground truth is whether it in retrospect was harmful or not) or in if it can match physician judgement in determining when a biopsy is needed (in which case the ground truth is if they physician ordered a biopsy)? You're asking the model to do the latter and grading it on the former.

Overall yeah probably specialized models make a ton of sense in a bunch of areas still but idk this does not seem like a good demonstration of that

yeah assuming no funny games the threat is real and the primary thing this CTF did while it was misconfigured and on the front page was convince people that the threat was not real. that is probably not an ideal outcome.

similarly to the guy from Trail of Bits talking about how it looked like a trap - apparently this PoC was not impossible for someone else to figure out, the window was not as long as you'd hope (again assuming no funny business), and the "trust me bro this exploit is real" messaging just kind of kills overall credibility and does more harm than good. As does a misconfigured CTF, potentially. People do not have a positive view of tech and especially AI right now and if there's a real threat then fueling the "it's just hype to boost stock price for IPO" narrative is maybe a bigger issue than the vulnerability itself no? Because you also tank your ability to get people to take any future vuln seriously.

Plus a broken CTF is apparently a good way to get people to not take a current vuln seriously while also working as a honeypot to get PoCs. Probably everyone involved with good intentions should make sure they're really thinking through all the implications (preferably ahead of time so there's a playbook). Meaning societal/psychological/narrative implications too and not just technical. Security of all fields should be deeply familiar with how human behavior is kind of critical, no? How many people reuse passwords? The AI industry is uniquely incompetent at this aspect. Like all the datacenter stuff where they make secret deals for tax breaks and cheap land with city councils under NDA and then are suprised that constituents react badly to being taken for a ride by an industry and elected officials they already don't trust. Like just from an optics perspective, charging for daybreak or similar frontier cyber-capable AI to defend from AI discovered/led vulnerabilities and attacks - y'all get how this looks like a protection racket right?

The worst outcome here is AI cyberwar/CVE-apocalypse hitting while the industry is treated as the boy who cried wolf even when there actually were previously wolves because everyone involved just sucks at comms and doesn't think things through and now has no credibility to organize a response.

From that latest update, your window seems to be closing now

OP just posted that there's a solve. And from a previous update thread apparently their first config was not actually exploitable? So ... seems like it is happening. Maybe not instantly, at least not if the site just remains down for hours because it's functionally DoSed, but it still is apparently exploitable.

The problem is that it's a very simple, older benchmark, and qwen was trained after fable 5. You need to test against similarly modern models so you aren't putting the models on uneven playing fields in terms of contamination. And for the stuff qwen is trained to be good at (agent stuff) opus 5.5 is light-years ahead of fable just because it's a newer model amd they RLed on it.

Though point taken that in a case where they didn't even bother making sure they were using the same qwen quants for every run or even test fable in their harness probably that is the least of the issues here.

Who said it's my assumption for code I generate? We're talking about how I evaluate code other people generated. If that wasn't my default assumption - then you should never trust anything I create because I would be naively trusting other people to maintain throwaway projects.

Are we losing our accumulated engineering best practice knowledge? Yes. Have you seen the quality of new hires recently? The ones who've cheated their way through school and don't know what a for loop is? People that have those skills aren't letting them atrophy but the chain of transmission is in crisis. Breaking news, sometimes things suck

I wonder if it's so circuitous because it buys the model more tokens for some sort of pseudo-reasoning before it locks in a token where there's no turning back. like it lets it push the logprobs way way way towards the right answer.

Plus using all this made up jargon and the unclear references to everything else (im referring to like the 'it' "XYZ is implemented using ABC. 123 referrs to it" you get in claudeslop) is very hard to understand but also kind of impossible to be wrong no? If it said "123 refers to ABC" but meant XYZ instead of ABC it's SOL but instead if it says "it" future tokens can ramble themselves out of being wrong thanks to the annoying slop ambiguity.

the problem with those hypothesis is that models don't in fact all talk the same (unless they're distilling claude). however if I unburden myself from having to be correct and allow myself to make up some more bullshit - what if "its not x its y" helps compute the right answer. Attending to that 'not x' could help point in the right direction in whatever dimension in latent space it needs to find the right y — and what if emdashes are proletariat filler tokens.

probably tho they're just RLing them to hell and back on code stuff and not paying attention to it and none of that is real lol. IIRC they rled opus 5.5 on talking more normal and it does and is also better than opus 5 so like with most things here, just another product decision on where to burn GPUs

That is not necessarily true if the codebase is written by AI unless the human chose to be involved. That was also not necessarily true before AI either. Human skill level is wildly variable

What's left to determine? Obviously AI written docs are a deal breaker for you. Seems very straightforward. If docs are ai written, move on. That's an immediate and obvious fundamental incompatibility you have with the project. Why are you still complaining about the docs being ai written getting in the way of making your decision when you've clearly already made it?

pretty much that's the default assumption yeah. Anything new is probably a throwaway. If there's anything cool in it - rebuild it to your own taste. There will be exceptions that do become long lived projects of course. probably even the same amount there were before. But there's going to be a few orders of magnitude more throwaways so on average any new piece of code is going to be a throwaway

at that point i just ask an ai questions about how the system works and hope the reasons for particular decisions can be reverse engineered from git history/pr comments etc.

its just been down for a while. At this point idk if anyone can tell if the site is unexploitable or it just turns out that people DoSed it before anyone with the exploit who wanted to use it could do it

So using this harness five qwen3.8-27b models roughly matched Fable 5 on a saturated coding benchmark from early 2025. No comparison to astra/6.1 sol/fable 5.1/opus 5.5. No comparison on a benchmark that reflects actual agentic work (and agentic behavior is what modern models optimize for anyway - probably makes qwen look worse than it is?). And also obviously using a very old benchmark like this:

> The updated release of the dataset with problems released between May 2023 and Apr 2025 containing 1055 problems

means essentially guarenteed contamination.

I have no idea if this harness is any good. Agressive orchestration could be interesting given how hard models are RLed on orchestration? But the methods used to evaluate this harness are not informative

Assuming you do have a working RCE - this is some weird game theory. Do it and potentially leak the exploit. Don't do it and you lose control of the narrative/lose your credibility/have people not take the risk seriously anymore (this comment is losing the fight with another comment for top comment, and the other comment is concluding that yeah this proof that the whole thing was overblown).

And then there's the third option that actually this server is unexploitable or there's no wallet or something and simultaneously you give someone the exploit and lose your credibility at the same time. Really no good way out huh?

Is highspeed a different model like a flash model or is that just a api/platform feature thats a normal fast mode? I can't tell from the announcement but I might just be missing something obvious?

they're pretty clearly framing it as holding back on the robot and letting is lose slowly to give everyone time to update their locks

Literally from the linked announcement:

> Kimi K3.1 Pro is a post-trained version of Kimi K3, with substantially larger post-training compute

If you want it to do that consistently why can't you put it in your Claude.md?