robkop

Worth taking a closer look at what they actually were able to formally verify. No doubt a massive achievement, however after seeing several demos poke holes through it I don’t think they’ve done the level of “formal verification“ that you may be inclined to think they have.

One of the main goals at the moment is to keep thinking human legible. You can imagine how much harder it’d be to do root cause analysis on the recent OpenAI event if we couldn’t even tell what they’re thinking.

Claude design's prompt is trivial to verify. They bundle it in the frontend bundle and send it on every network request.

CoT legibility largely disappears which is quite concerning from a safety perspective

Just saying you’re not alone, very surprised by the reception given how brutally sloppified the OP is.

Interesting problems space but I hope the author just gives dot points next time rather than bloating it and losing most of its meaning.

Could you please elaborate a bit more for my understanding?

What in particular about this method breaks correct token boundaries?

On my first read I read your comment as there are special tokens that require multiple tokens to emit, hence you can't get certain tokens emitted alone - but I don't think that's what you're getting at on a second read?

Interesting that you've found similarities between "d" and the hidden tokens for opening an xml tag, pressing caps lock and the other hidden tokens of note. I haven't run into any trouble extracting "d" tokens, is it a particular model that you see create that pattern?

I use bedrock with 1M context every day. Not sure this is right

A lot of enterprises were doing that but now they hit the 150 user limit on Claude and are paying seat+api rates.

Codex is still going strong but it’s hard to imagine they won’t do similar eventually.

So now im honestly hearing a lot more folk stick it out with cursor while waiting for the dust to settle.

There’s a lot of tradeoffs to play with, those inference ASICs may not carry the gradient but they are still optimised for larger batches and to run any model. They need enough memory for the weights, wide batch inference, and ideally leftovers for kv cache efficiency.

For personal inference you’re given a lot more room to play in - much of it poorly explored today - enough to concern an argument of cost advantages evaporating

You can ablate surprisingly large chunks of a model with near to no effect, you can try this easily - download an open weight model in torch.

Obviously it’s not ideal but you could likely have single digit % of all weights affected and still have a useful model (many caveats here: e.g. locality of damaged weights matters, distribution of errors matters, fail high/low matters, …)

I can’t speak for the states, but in AU I clearly see a massive displacement of undergrad and junior roles (only in AI exposed domains).

I say this as both someone who works with many execs, hearing their musings, and someone who no longer can justify hiring junior roles themselves.

Irrespective of that; if we take this strategy of only taking action once it is visible to the layman - our scope of actions available will be invariably and significantly diminished.

Even if you are not convinced it is guaranteed and do not believe what myself and others see. I would ask you is your probability of it happening now really that close to 0? If not then would it not be prudent to take the risk seriously?

one of their highlights with mythos was it's ability to generate new puns

I took a look and honestly they're the first AI puns that aren't bad

Times are changing

We’ve got a long way to go in optimising our environments for these models. Our perception of a terminal is much closer to feeding a video into Gemini than reading a textbook of logs. But we don’t make that ax affordance at the moment.

I wrote a small game for my dev team to experience what it’s like interacting through these painful interfaces over the summer www.youareanagent.app

Jump to the agentic coding level or the mcp level to experience true frustration (call it empathy). I also wrote up a lot more thinking here www.robkopel.me/field-notes/ax-agent-experience/

Rumours say you do something like:

  Download every github repo
    -> Classify if it could be used as an env, and what types
      -> Issues and PRs are great for coding rl envs
      -> If the software has a UI, awesome, UI env
      -> If the software is a game, awesome, game env
      -> If the software has xyz, awesome, ...
    -> Do more detailed run checks, 
      -> Can it build
      -> Is it complex and/or distinct enough
      -> Can you verify if it reached some generated goal
      -> Can generated goals even be achieved
      -> Maybe some human review - maybe not
    -> Generate goals
      -> For a coding env you can imagine you may have a LLM introduce a new bug and can see that test cases now fail. Goal for model is now to fix it
    ... Do the rest of the normal RL env stuff

I get this at least once a week. And then once you have to dig in and understand the full mental model it’s not really giving you any uplift anyway.

I will say that doing this for enough months has made my ability to pick up the mental model quickly and to scope how much need to absorb much quicker. It seems possible that with another year you’d become very rapid at this.

I added a "Human" LLM provider to my local OpenCode a few months ago as a joke, and it turns-out acting as a LLM is quite painful. But it massively improve my agent harnesses dev skills.

So I thought I wouldn't leave anyone out! I made a small oss game - You Are An Agent - youareanagent.app - to share in the (useful?) frustration

It's a bit ridiculous. To tell you about some entirely necessary features, we've got: - A full WASM arch-linux vm that runs in your browser for the agent coding level - A bad desktop simulation with a beautiful excel simulation for our computer use level - A lovely WebGL CRT simulation (I think the first one that supports proper DOM 2d barrel warp distortion on safari? honestly wanted to leverage/ not write my own but I couldn't find one I was happy with) - A MCP server simulator with full simulation of off-brand Jira/ Confluence/ ... connected - And of course, a full WebGL oscilloscope music simulator for the intro sequence

Let me know what you think!

Code (If you'd like to add a level): https://github.com/R0bk/you-are-an-agent (And if you want to waste 20 minutes - I spent way too long writing up my messy thinking about agent harness dev): http://robkopel.me/field-notes/ax-agent-experience/

It's a fair question - I think the fact that they hold abilities (read 200k tokens instantly, can clone themselves, ...) that we don't would suggest they will have quirks and differecnes.

What downstream implication that will have on a AX sense is certainly arguable, but I would put forward that we're already seeing it with effective harnesses such as Claude Code. The experience the agent has there is quite different to how you'd build an IDE for a human.

Can you elaborate? I would have thought the main driver for the price of a service is the labor?

Does that cost to serve multiple stay the same when conventional sites are forced to shovel ai into each request? e.g. the new google search

I’ve heard too many rumors that much of that adoption is from copying ms i.e. bundling gemini into their office suite

Occam's Razor - this complexity arises from the human nature to try and build consistent abstractions over complex situations. It's exactly what we do in software too. To an outsider it's going to look nonsensical.

I want to share a thought experiment with you - atop an ancient Roman legal case I recall from Gregory Aldrete - The Barbershop Murder.

Suppose a man sends his slave to a barbershop to get a shave. The barbershop is adjacent to an athletic field where two men are throwing a ball back and forth. One throws the ball badly, the other fails to catch it, and the ball flies into the barbershop, hits the barber's hand mid-shave, and cuts the slave's throat-killing him.

The legal question is posed: Who is liable under Roman law?

- Athlete 1 who threw the ball badly

- Athlete 2 who failed to catch it

- The barber who actually cut the throat

- The slave's owner for sending his slave to a barbershop next to a playing field

- The Roman state for zoning a barbershop adjacent to an athletic field

Q: What legal abstractions are required to apply consistent remedies to this case amongst others?

Opinion: You'd need a theory of negligence. A definition of proximate cause. Standards for foreseeability. Rules about contributory fault. A framework for when the state bears regulatory responsibility. Each of those needs edge cases handled, and those edge cases need to be consistent with rulings in other domains.

Now watch these edge cases compound, before long you've got something that looks absurdly complex. But it's actually just a hacky minimum viable solution to the problem space. That doesn't make it fair that citizens bear the burden of navigating it - but the alternative is inequal application of the law

For those curious about the "consistent principle of law" here - SCOTUS wrestled with nearly exactly this question in Free Speech Coalition v. Paxton earlier this year, and effectively emboldened more of these laws.

Previously the Fifth Circuit had relied heavily on Ginsberg v. New York (1968) to justify rational basis review. But Ginsberg was a narrow scope - it held that minors don't have the same First Amendment rights as adults to access "obscene as to minors" material. It wasn't about burdens on adults at all. Later precedent (Ashcroft, Sable, Reno, Playboy) consistently applied strict scrutiny when laws burdened adults' access to protected speech, even when aimed at protecting minors.

In Paxton the majority split the difference and applied intermediate scrutiny - a lower bar than strict - claiming the burden on adults is merely "incidental." Kagan had a dissent worth reading, arguing this departs from precedent even if the majority won't frame it that way. You could call it "overturning" or "distinguishing" depending on how charitable you're feeling.

The oral arguments are worth watching if you want to understand how to grapple with these questions: https://www.youtube.com/watch?v=ckoCJthJEqQ

On 1A: The core concern isn't that age-gating exists - it's that mandatory identification to access legal speech creates chilling effects and surveillance risks that don't exist when you flash an ID at a liquor store.

Note: IANAL but do enjoy reading many SC transcripts

Interesting question - how much will end up as sound, or in the ever smaller tail of things like storing a bit in flash memory?

I remember you did some reverse engineering when they released agent, does it not feel quite similar to you?

I know they didn’t dynamically scan for new skill folders but they did have mentions of the existing folders (slides, docs, …) in the system prompt

Hasn’t ChatGPT been supporting skills with a different name for several months now through “agent”?

They gave it back then folders with instructions and executable files iirc

You could dual brand as vibe-npm, only install packages that are in your models training dataset

> Are there specific benchmarks that compare models vs themselves with and without scratchpads? High with:without ratios being reasonier models?

Yes, simplest example: https://www.anthropic.com/engineering/claude-think-tool

Being skeptical of all the numbers I see - it still seems instagram is on roughly even footing with TikTok for upcoming generations.

I don’t doubt they may destroy their own product (like google search) but I do think it’s going to take a long long time

There is much missing from this prompt, tool call descriptors is the most obvious. See for yourself using even a year old jailbreak [1]. There’s some great ideas in how they’ve setup other pieces such as cursor rules.

[1]: https://gist.github.com/lucasmrdt/4215e483257e1d81e44842eddb...