vishvananda

For reference I blogged about the previous version here which was ahead of gcc in compile time and produced 50% slower compiled code here: https://medium.com/@vishvananda/agentic-c-compiler-redux-5d1... I haven’t updated with the most recent results but the current v5 reference has the perf stated above

It compiles about 7x faster both at -O0 and -O3. My optimization is not quite on par but it is really close. Compiled code is about 5% slower.

I’ve been coming around to the same conclusion. We are really just scratching the surface of what we can do with these new tool1. The windmill I have been tilting at is creating a c++ compiler[1] and the most recent version is able to compile about 7x faster than gcc[2]. I don’t know that this is super-useful because people aren’t writing much in c++ these days, but as the article mentions, faster rust compilation would accelerate agentic coding considerably. I’ve just started tackling rust and I don’t have a confirmed result yet but early testing makes me think I can find a 4x+ speed up here.

[1]: https://github.com/vishvananda/cppgm-extended

[2]: caveats here include only c++11, only x86, and potential for bugs, but 7x is pretty darn significant even with caveats

I do some pretty insane agentic work running constantly. I’m currently burning through multiple $200 accounts every week. I regularly spend $10-$20k worth of tokens a month. Most of it has been going to my experimental c++ compiler project

The reason people aren’t freaking out is because most people are using heavily subsidized subscriptions.

I tried the cheapest provider on openrouter and burned through $50 in a few days. Quality was ok, seems slightly above Luna quality perhaps? But that $50 is 1/4 of my codex subscription where I could have burned that many tokens or more using Astra within my weekly reset.

This won’t last forever but as long as the frontier labs are subsidizing this heavily the open models won’t matter.

I've been doing a lot of experimenting with this with a c++ compiler project[0]. I definitely experienced a lot of challenges, but I do believe it is possible to end-up in a consistent space by putting in some very careful guardrails. I covered some of the things i learned in the follow on blog posts. My more recent runs have been producing better code after continuing to improve harnesses, tests, and prompts. New blog post coming soon on v2 and v3 reruns. I'll be kicking off some v4 runs soon. Here is a teaser showing the later runs: https://storage.googleapis.com/ralph-run-viewer-zippy-960/in...

[0]: https://medium.com/@vishvananda/i-spent-2-billion-tokens-wri...

From harness? Because people expect a squishy set of things from a harness that is different from what I end up building. I end up with a rigid internal structure that the harness uses in-turn (tests with clear error messages, tools, etc.) and a matched rigid external structure that drives the turn tracking progress and deterministically handles the overall progress. You could call that whole thing a harness but that makes the definition muddy and hard to talk about. So scaffold or skeleton seems more appropriate. The harness constrains the agent. The matched endoskeleton and exoskeleton gives it structure.

I’m definitely on the deterministic code train as well. All of my success for long running tasks has been around wrapping the agentic harness (cc, codex-cli, etc.) in a deterministic workflow with deterministic gates. We need a name for this outer layer. In my mind that is the true harness because it constrains the agents failure mode. I think flow engineering has been proposed. Maybe it’s the agentic exoskeleton?

For me it was earlier this year when I started dusting off some old stalled projects and had an agent work on them. In a few days I:

* Built a clone of the Alpha Zero implementation[1] my team built at oracle

* Ported my hobby NES emulator from javascript to rust[2] (this actually took less than 30 minutes and worked on the first try)

* Implemented all of the lessons from the C++ Grandmasters Challenge (which eventually led to a complete c++ compiler[3])

The thing that flipped the switch was using it to build things that I actually put sweat-equity in to previously. I knew how hard these things were to build, so it landed in a way that other projects had not.

[1]: https://medium.com/oracledevs/lessons-from-implementing-alph...

[2]: https://github.com/vishvananda/popeye

[3]: https://medium.com/@vishvananda/i-spent-2-billion-tokens-wri...

I've been experimenting quite a bit with long-horizion agentic coding[1] and I have also noticed that agents seem to perform worse when forced into certain architectural patterns. I have found that is a bit better when including the constraints along the way instead of adding them after the fact. There seems to be a side-effect I have been calling "calcification", where a pattern starts appearing in the codebase and the agent follows the pattern to the point where it dominates the context and becomes self-reinforcing. This could potentially be a strength or a weakness for existing code bases depending the codebase quality. I will have more insights on this soon as more from-scratch runs conclude that include architectural guidance from the beginning.

[1]: https://medium.com/@vishvananda/i-spent-2-billion-tokens-wri...

I think there is a flow in most organizations from:

llm -> prompt -> result

llm -> prompt + prompt encoded as skill -> result

llm -> prompt + deterministic code encoded as skill -> result

I do think prompting to generate code early can shortcut that path to deterministic code, but we're still essentially embedding deterministic code in a non-deterministic wrapper. There is a missing layer of determinism in many cases that actually make long-horizon tasks successful. We need deterministic code outside the non-deterministic boundary via an agentic loop or framework. This puts us in a place where the non-deterministic decision making is sandwiched in between layers of determinism:

deterministic agentic flows -> non-deterministic decision making -> deterministic tools

This has been a very powerful pattern in my experiments and it gets even stronger when the agents are building their own determinism via tools like auto-researcher.

This is actually a very interesting moment to potentially overcome network effects, because more and more code is going to be written by agents. If a crdt approach is measurably better for merging by agent swarms then there is incentive to make the switch. It also much easier to get an agent to change its workflow than a human. The only tricky part is how much git usage is in the training set so some careful thought would need to be given to create a compatibility layer in the tooling to help agents along.

I share the concern based on the current productivity expectations but I did stumble across something recently which makes me feel a lot better. There was a change in the tax code that coincides with the beginning of the post-pandemic layoffs[1]. This was changed back last month by the BBB which likely means a bunch of new R&D spend for big tech. I think this is why we are seeing intense M&A activity and if we can keep the AI hype under control it will probably lead to new hiring as well.

[1]: https://qz.com/tech-layoffs-tax-code-trump-section-174-micro...

I am by no means an AI skeptic. It is possible to encode all sorts of things into instructions, but I don’t think the future of programming is every individual constructing and managing artisan prompts. There are surely some new paradigms to be discovered here. A code locking interface seems like an interesting one to explore. I’m sure there are others.

I'm really baffled why the coding interfaces have not implemented a locking feature for some code. It seems like an obvious feature to be able to select a section of your code and tell the agent not to modify it. This could remove a whole class of problems where the agent tries to change tests to match the code or removes key functionality.

One could even imagine going a step further and having a confidence level associated with different parts of the code, that would help the LLM concentrate changes on the areas that you're less sure about.

Heroku is turning the 12 factor manifesto into a community project and modernizing it. I posted my thoughts in a blog[1], and I'd love to hear what other people think!

[1] https://medium.com/@vishvananda/the-rebirth-of-twelve-factor...

This is something I’ve been arguing for for a while[1]. I called it a “Framework Knowledge Base”. I think it needs to go a bit further and include specific code examples, especially for newer bits that are not in the training set. Ultimately RAG or even fine tuning might be better than a system prompt. [1]: https://devops.com/the-rise-of-coding-assistants-superchargi...

Doesn’t this already exist as https://www.npmjs.com/package/dotenv-cli ?

I've followed Caleb on YouTube for a while, due to his MtG content. He is PhD in Optical Sciences that makes some very interesting AI art. He has created a new card game with hand curated AI art and some interesting rules aimed at solving a bunch of the problems around existing collectible card games. He wrote an essay[1] about his design goals which is fascinating. [1] https://calebgannon.com/2023/07/08/the-making-of-algomancy/

Just reran the benchmarks from 10 years ago, python is only 37X slower than C on the benchmark now, and the go version is running faster than the C version. Python still has big productivity wins of course...

Python is extremely slow for some tasks. I was surprised to discover how slow when I ran some benchmarks, despite having used python for many years at the time. It has been improving lately, but here is a blog post I made on the topic quite a few years ago that has some interesting comparisons: https://gist.github.com/vishvananda/7a2f1942d0e9ffff4093

This is definitely an intriguing line of thinking. I too am constantly appalled by the complexity we introduce into the things we build, but I suspect a lot of it has to do with human issues that can't be solved by better technology. That said, I'm very curious to see what tractor ends up looking like.

I'm surprised that neither the documentary nor the review gets into the legacy of OpenStack. I may be biased, but it seems to me that a huge amount of the success of kubernetes is directly attributable to OpenStack.

First, OpenStack paved the way for a bunch of companies to invest real money in working together to compete with AWS. Second, there was massive turnover in ~2013 in open source contributors from OpenStack to Kubernetes. I wouldn't be surprised if a good 50% of the kubernetes community was inherited directly from OpenStack.

In this particular case, some of the runs from iFit instructors are actually quite good, and it cool that it adjusts the speed and incline to match the instruction. Probably not worth the extra $$$$ but it is pretty cool. But now I also want to be able to watch regular videos. I usually walk outdoors for an hour a day to get my 10,000 steps in, and the Chicago winter makes that tough, so I'm thinking an hour of walking on the treadmill while i catch up on my favorite shows might be a good substitute.

Thank you for this. I was hoping someone had figured it out but I hadn't found this yet.

I hadn't been following the "Right to Repair" movement too closely until I discovered that NordicTrack recently locked their users out of Privileged Mode[0], which makes it virtually impossible to install 3rd party software like Netflix on my treadmill. I just want to watch a video while I exercise!

[0]: https://www.wired.com/story/nordictrack-ifit-treadmill-privi...

You could always use a pointer for this, but admittedly it is pretty ugly when compared to a true optional type:

    func f(s *string) string {
        ret := ""
        if s == nil {
            ret = "default"
        } else {
            ret = *s
        }
        return ret
    }

    func main() {
        s := "foo"
        fmt.Println("Hello, " + f(&s))
        fmt.Println("Hello, " + f(nil))
    }

While the particular issue of network namespaces and locking the os thread was fixed, there is still c code that must run before the go runtime starts to work around the issue that you cannot do some of the necessary nsenter calls once you have started additional threads. The c code to make runc work is encapsulated in libcontainer/nsenter[0]

[0]: https://github.com/opencontainers/runc/tree/master/libcontai...

Go is actually a really poor choice for the container runtime because much of the container setup cannot be done from multithreaded code[0], so it has to be done in C before the go runtime initializes. I do think rust is a better choice for this layer than C because there are still security risks, but getting rid of Go for this layer is a win. I'm not sure why RH chose to rewrite it in C rather than using rust[1].

[0]: https://www.weave.works/blog/linux-namespaces-and-go-don-t-m... [1]: https://github.com/drahnr/railcar

I was the main developer with a little support from the rest of my team. I moved on a couple of years ago and my team was RIFed a few months later, so likely they don't have anyone left to work on it.