If found that stack languages are a massive weak spot for LLMs. It requires accurate tracking of what is on the stack and for some reason that doesn't really work. So even the best models end up making the same basic mistakes over and over again.
So for example I tried to have LLMs write xml parsers from scratch in Q3 2025, but they kept getting confused and midstream would migrate from c# to xml (that is their solution was half c# half xml file). So thinking in a tree data structure was beyond them.
Then Q1 2026 they were able to write parsers including parsers for languages where I specified with a bunch of examples.
But I tried specifying a language that I thought would require storing the AST in a graph structure. Everytime the LLM would instead parse into a tree and then do post processing on the tree to get the right output. So perhaps graph structures are (were?) beyond them.
I haven't investigated forth for a while, but it's been on my to-do list. I'll have to bump it up some because it sounds like I can kill two birds with one stone here.
I do kind of wonder if tree space is a local maximum, though. You can do quite a lot of things in that space and most of the code that I've read that leans on some sort of graphical thinking (like heavy usage of global variables that effect the execution path all over the call graph) is a less than ideal substrate to have to think through.
[One of the problems with investigating this is that I can't just ask it to solve a problem that is obviously just dijkstra because that's in the training data.]
Do you mean stack-based source code languages like Forth or PostScript, or directly writing machine instructions for a stack-based ISA/VM like the JVM?
If the training set were loaded with Forth, LLMs would be knocking it out of the park.
In reasoning mode, if Forth were in high demand, I expect a modern LLM would blast out a python script to help it reason through the stack. That's how they seem to handle most problems that they don't innately understand.
That is what I thought initially as well, but I'm no longer sure that that is the case. Forth requires a mental model of what is going on on the stack and that mental model can get pretty complex. There is a lot more 'scope' in Forth than in most modern languages, you can reach pretty much any memory location and fetch any word from the stack, effectively the whole machine is in scope.