phamilton

> Several dollars per million tokens

The flagship, glm-5, is $1/M input tokens. glm-4.7 is $0.60/M input tokens.

Not yet. But what's the actual goal here? It's not to have a native Wave Race 64. It's to improve my intuition around what sort of tasks can be worked on 24/7 without supervision.

I have a hypothesis that I can verify the result against the original ROM. With that as the goal, I believe the agent can continue to grind on the problem until it passes that verification. I've seen it in that of other areas, but this is something larger and more tedious and I wanted to see how far it could go.

Step 1: Decompile into C that can be recompiled into a working ROM. In theory, it could be compiled into the same ROM that we started with. Consistent ROM hash is the main success criteria for the OoT decompilation project. Have it grind until it succeeds.

Step 2: Integrate libultraship. Launching the game natively is the next criteria. Then ideally we could do differential testing on a frame by frame basis comparing emulated vs native.

Step 3: Semantic documentation of source. If it gets this far, I will be very impressed.

This is absolutely an experiment. It's a hard problem with low stakes. There a lot to learn from it.

It's using about 100M input tokens a day on glm 4.7 (glm 5 isn't available on my plan). It's sticking pretty close to the throttling limits that reset every 5 hours.

100M input tokens is $40 and anywhere from 2-6 kWh.

Certainly excessive for my $3/month.

As an experiment, I set it up with a z.ai $3/month subscription and told it to do a tedious technical task. I said to stay busy and that I expect no more than 30 minutes of inactivity, ever.

The task is to decompile Wave Race 64 and integrate with libultraship and eventually produce a runnable native port of the game. (Same approach as the Zelda OoT port Ship of Harkinian).

It set up a timer ever 30 minutes to check in on itself and see if it gave up. It reviews progress every 4 hours and revisits prioritization. I hadn't checked on it in days and when I looked today it was still going, a few functions at a time.

It set up those times itself and creates new ones as needed.

It's not any one particular thing that is novel, but it's just more independent because of all the little bits.

Intelligence per token doesn't seem quite right to me.

Intelligence per <consumable> feels closer. Per dollar, or per second, or per watt.

Maybe I expressed that clumsily.

With historical development, investing in hypotheticals can be wasteful. Make the fewest assumptions until you get real user feedback.

With AI, we make more decisions upfront. Being wrong about those decisions has a low cost because the investment in implementation is cheap. We are less worried about building the wrong things because we can just try again quickly. It's cheap to be wrong.

The more decisions we make upfront, the more hands-off we can be. A shallow set of decisions might stall out earlier than a deeper set of decisions could. So we need to think through things more deeply. So we need to think more.

Not as much upfront. I had plenty of opportunities to adjust and correct along the way. With AI, the cost of not thinking upfront is high and the cost of being wrong in upfront decisions is low, so we bias towards that.

But beyond that, I have been thinking deeply about AI itself, which has all sorts of new problems. Permissions, verification, etc.

I think harder because of AI.

I have to think more rigorously. I have to find ways to tie up loose ends, to verify the result efficiently, to create efficient feedback loops and define categorical success criteria.

I've thought harder about problems this last year than I have in a long time.

It all comes back to "Do more because of AI" rather than "Do less because of AI".

Getting back into coding is doing more. Updating an old project to the latest libraries is doing more.

It often feels ambiguous. Shipping a buggy, vibe-coded MVP might be doing less. But getting customer feedback on day one from a real tangible product can allow you to build a richer and deeper experience through fast iteration.

Just make sure we're doing more, not less, and AI is a wonderful step forward.

I tried to ask Gemini about the blog content and it was unable to access the site. It was blocked and unable to discover the API in the first place.

The inf1/inf2 spot instances are so unpopular that they cost less than the equivalent cpu instances. Exact same (or better) hardware but 10-20% cheaper.

We're not quite seeing that on the trn1 instances yet, so someone is using them.

It successfully got through the captcha at https://www.google.com/recaptcha/api2/demo

Just tried it in an existing coding agent and it rejected the requests because computer tools weren't defined.

Does that just do Database Level Conflict Checking?

I'm sad this project fizzled: https://github.com/losfair/mvsqlite

It had the fascinating property of being a full multi-writer SQL engine when using Page Level Conflict Checking. Even more fascinating: every single client itself becomes a writer.

That's a different problem. To quickly sort a nearly sorted list, we can use insertion sort. However the goal is to make progress with as little as one iteration.

One iteration of insertion sort will place one additional element in its correct place, but it leaves the unsorted portion basically untouched.

One iteration of bubble sort will place one additional element into the sorted section and along the way do small swaps/corrections. The % of data in the correct location is the same, but the overall "sortedness" of the data is much better.

This feels similar to when I heard they use bubble sort in game development.

Bubble sort seems pretty terrible, until you realize that it's interruptible. The set is always a little more sorted than before. So if you have realtime requirements and best-effort sorting, you can sort things between renders and live with the possibility of two things relative close to each other appearing a little glitched for a frame.

I think we should focus less on API schemas and more on just copying how browsers work.

Some examples:

It should be far more common for http clients to have well supported and heavily used Cookie jar implementations.

We should lean on Accept headers much more, especially with multiple mime-types and/or wildcards.

Http clients should have caching plugins to automatically respect caching headers.

There are many more examples. I've seen so much of HTTP reimplemented on top of itself over the years, often with poor results. Let's stop doing that. And when all our clients are doing those parts right, I suspect our APIs will get cleaner too.

(not trolling) Would that undefined behavior have occurred in idiomatic rust?

Will the ability to use AI to write such a solution correctly be enough motivation to push C++ shops to adopt rust? (Or perhaps a new language that caters to the blindspots of AI somehow)

There will absolutely be a tipping point where the potential benefits outweigh the costs of such a migration.

My LLM workflow involves a back-and-forth clarification (verbose input -> LLM asks questions) that results in a rich context representing my intent. Generating comments/docs from this feels lossy.

What if we could persist this final LLM context state? Think of it like a queryable snapshot of the 'why' behind code or docs. Instead of just reading a comment, you could load the associated context and ask an LLM questions based on the original author's reasoning state.

Yes, context is model-specific, a major hurdle. And there are many other challenges. But ignoring the technical implementation for a moment, is this concept – capturing intent via persistent, queryable LLM context – a valuable problem to solve? I feel it would be.

If we can detach content and presentation, then the reader can choose tone and length.

At some point we will stop making decisions about what future readers want. We will just capture the concrete inputs and the reader's LLM will explain it.

Jokes aside, this happens all the time.

I have it write doc strings. I later ask it to explain a section of code, wherein it uses the doc strings to understand and explain the code to me.

A less lossy way to capture this will probably emerge at some point.

I can run the Qwen 3 0.6B model directly on my 3-year-old phone, and it can rewrite text and help me clearly explain my views. Even with new models possibly being less open and subsidized options drying up, we still have useful open models available for free on consumer hardware.

Source: I used it to write this post.

We used to have a "2 approvals" policy on PRs. It wasn't fully enforced, it was a plugin to Gitlab we built that would look for two "+1" comments to unhide the merge button.

I used to create PRs and then review my own code. If I liked it, I'd +1 it. If I saw problems, I'd -1 it. Other engineers would yell at me that I couldn't +1 my own code. When I showed them PRs that had my -1 on it, they just threw their hands up and moved on, exasperated.

I've carried that habit of reviewing my own code forward, even though we now have real checks that enforce a separate reviewer. It's a good habit.

Other feedback: allow permalinks for a given day's prompt. I expect to keep a collection of clever techniques and share with my team.

I would put attempt history above the leaderboard. Having to scroll past it to see the results of my submission makes it hard to not peek.

A nice addition to this I've seen twice now is a slack channel (via their personal emails) with continuing employees willing to help them practice interviewing and share their professional networks to help them find their next role.

I suspect this will indeed be part of it, but it won't work with today's AIs on today's codebases.

Models will improve, but also I predict code style and architecture will evolve towards something easier for machine review.

Agents could generate more PRs in a weekend than my team could code review in a month.

Initially we can absolutely just review them like any other PR, but at some point code review will be the bottleneck.