175 karma · joined June 14, 2026
I was only able to do that after I had solved multiple related problems in different places and started introducing subtle bugs by accident / had difficulty detecting all edge cases
I've noticed whenever I use LLMs they introduce the same kind of thing but at much smaller scales than I would. They often suggest solving the wrong problem when I prompt them to diagnose specific bugs too. Usually opting for a shortcut that introduces its own issues and ironically calling the proper direction "too complex" when it's really not.
I am working on a compiler for an OCaml inspired language (I am on a sabbatical) and used them a lot in the early phases to check my ideas and search for prior work. I use them less now (mostly debugging at this point), but have made use of LLMs to write some pieces like source map generation, a couple of small and well known algorithms that were new to me, and an analyzer to dump a schema from a typescript module. Other times I've tried to use it for other things, I have ended up regretting it because of subtle bugs
I don't think it made the design phase any shorter but it definitely helped save me energy in researching and made that period feel less like a fever dream. Design phases can be draining for me so that was welcome
I started using harnesses because they are good for when something breaks and it's not trivial to investigate so I'll have the agent tell me what's happening, then using that to produce my own change
I am considering .NET for one of my compilers backends because of the reified generics. .NET can even pass around an object with generic methods that get specialized via JIT at runtime each time it sees a new data type (with reference types sharing implementations). Which also ties back to having true value types
It's a shame it took so long for .NET core to come around because even today the platform carries a reputation for being windows first which hasn't really been true for many years now
I am very skeptical of these workflows, but I also use LLMs regularly. I specifically use them because they are better than me at sifting through massive amounts of complex information. They are also quite, uh, sketchy, for things that are significantly easier. A total mixed bag in my experience that ultimately is useful, and I will continue to use
I use LLMs all the time to help me diagnose bugs and work through my designs, but again and again, I am super unimpressed by their coding abilities. I can see how in some cases with a proper harness they probably do a decent job at certain tasks, but almost everything I try to do, they flail.
I tried having them work on a LSP. The fact I got a one shot half working autocomplete based on my existing work was cool, but again, they flailed on incredibly simple things like file path normalization / converting from a URI and I had to rewrite a decent amount of code. I don't think I saved any time
People keep throwing this out there but I keep wondering where are the receipts? I am seeing less interesting software released, anecdotally I know, since AI has taken hold, than before.