A Forth Apologia (1988) | Hacker News Reader

A Forth Apologia (1988)

holonforth.com

79 pointsnyc11154 comments

Forth is a mindset as much as a programming language. It is a language for minimalists. If you can solve your problem efficiently in Forth, you can be confident that your solution will work in essentially any computing environment. Unlike C, the core of Forth is small enough that a single individual can implement it in a few days at most. This is incredibly liberating because you will realize that you will never be seriously hindered by the tools provided to you because the Forth way is to just write your own tools. Of course this goes against the modern programming culture which has become extremely maximalist and makes you dependent on complex tools like VSCode + LSP in order to be productive. There is another way.

The language itself leaves a lot to be desired, in my opinion, but the core idea of having a small extensible compiler around which one can build efficient problem specific DSLs is brilliant. Unfortunately I think it is too easy to get bogged down in the bad parts (or at least parts that appear bad to our modern programming sensibilities). For a good overview of what makes Forth special, I recommend Thinking in Forth https://thinking-forth.sourceforge.net even if you never expect to program in Forth. It was written in the 80s but it still holds up. Read the first 80 pages or so carefully and I can almost promise that it will change the way you think about programming in a good way.

Once you have internalized the Forth mindset, you don't need to actually write Forth code. But you may be grateful that Forth taught you how to find your optimal programming language that exists somewhere inside of yourself.

Forth is an important skill for a zombie apocalypse. You can create a simple computer, with a simple instruction set, and get a high level programming language in a short time :)

interestingly enough, while this seems like an obviously good application for forth, i have never actually seen someone get to a working c compiler this way

This recent project has a C compiler: https://git.sr.ht/~vdupras/duskos

https://lists.sr.ht/~vdupras/duskos-discuss/%3C69cfdb97-49d6...

hey that sounds great

i will take a look

A Forth programmer who bootstraps their own Forth probably isn't that excited about having a c compiler.

maybe i don't qualify as a forth programmer then

well, that's unfair, because https://github.com/kragen/stoneknifeforth definitely doesn't qualify as a forth, it doesn't have immediate words

Back in the day, Forth wasn't so much software, as an amazing idea for how to make software for platforms where you had no tools/or terrible tools, which was most personal computers or embedded environments. Once you groked how Forth worked, it was quite practical to put together a language kernel in ASM or machine code and you were off to the races.

i think maybe a lot of the difficulty i had understanding forth was that i mistakenly thought of it as a programming language, and considered as a programming language it's not that great, much like bash

Forth is definitely a language, and depending on what you have available, it can be a great option. These days, when you can download gcc cross compilers for almost anything, there's less need to roll-your-own Forth, but it's still pretty interesting to study.

forth definitely has a language in it but i don't think the language is what led people to prefer it to, e.g., c or a nice macro assembler

(i'd say 'leads' but there don't seem to be a lot of people doing things in forth these days; duncan's 01988 estimate of 50000 'serious' forth programmers seems two orders of magnitude higher than seems likely today)

i think the appeal is that in 16k or 64k you get a repl which (unixlike) doubles as a scriptable ui, a sort of source-level debugging, interactive variable inspection (and i/o port poking, memory dumping, etc.), instant turnaround for changes (no 'my code's compiling'), multithreading, virtual memory, an assembler and disassembler, an extensible compiler with a turing-complete macro system, and dramatically better code density than machine code

all of that makes a pretty appealing package, especially for certain kinds of tasks, and it's only loosely coupled to the language qua language

as duncan points out, part of the appeal is that the development environment is a lot more integrated than something like turbo pascal

I have tried Forth and failed many times.

The biggest problem for me was the amount of effort needed to keep track of "local" variables.

Was I missing something? Is there a way to let Forth keep track of where on the stack "foo" lives if it is needed several times in a subroutine instead of always shuffling the stack or counting the elements on it?

> Was I missing something? Is there a way to let Forth keep track of where on the stack "foo" lives if it is needed several times in a subroutine instead of always shuffling the stack or counting the elements on it?

I'm not a Forth novice, much less an expert, but my impression is that the answer is that if you spend much time thinking about the location of variables, then you're "doing it wrong"—to steal the phrase, trying to write C in Forth, rather than to write Forth. My understanding is that the emphasis is on building highly domain-specific languages out of many small words, each of which is re-factored so much that it never needs to keep much track of any, or much, shuffling of variables on the stack.

So basically "use it as a language for combinators", right?

But combinator languages are pretty hard to read no matter what one does, don't they?

forth is ruthlessly imperative

    x = a[n-1];
    n += y * x;
    plot(x, y, n);

forth:

    : foo n @ 1- a @ x ! ;  \ ‡
    : bar y @ x @ * n +! ;
    : baz x @ y @ n @ plot ;
    : quux foo bar baz ;

naming each line of code allows you to do things like

    bar n ?

to see what n is after you bar

in exchange for the somewhat rebarbative syntax and lack of typing you get turing-complete macros (with full compiler access, without the pitfalls of string-replacement macros) and an interactive repl that fits into almost the smallest microcontrollers

because you have total freedom to redefine the syntax you can construct whatever dsl you prefer

it is nothing like combinatory logic except that you can occasionally write a point-free function

______

‡ this assumes a is defined as something like this

    : array create  cells allot
      does>  swap cells + ;
    20 array a

so that 3 a computes an &a[3] suitable for @ and !, etc.

Was I missing something?

yes

use variable or value for variables, not the stack

the stack is for expression evaluation

roughly everyone has this problem at first

if you'd put it in a local variable in c or js, 85% of the time you should put it in a variable in forth. start with 100% or 110% and scale back from there

yes this means your words aren't reentrant by default

this is ok, most subroutines aren't recursive

never shuffle the stack or count the elements in it. if you find yourself doing that, put the elements you wanted to access in a variable or value

then you can access them by name. this is much easier

the possibility of shuffling the stack is an attractive nuisance, like a tar pit. an occasional dup or swap or r@ is okay, but start by not using them

literally write square as

    variable x
    : square x ! x @ x @ * ;

    value x
    : square to x  x x * ;

if that's what it takes to stop tying your code in knots

it is ok to name a variable x, its scope only extends to the next definition of an x. it's not like in c where every x needs to either be local to a subroutine or a single global unique x used by the entire program (or at least source file)

later you can graduate to sometimes replacing one local variable with the return stack and another one or occasionally two with the operand stack, but it is better for your code to be kind of boring and verbose than an incomprehensible soup of 2dup -rot tuck

forth is maybe not better than c but there is no reason to make it worse

This is roughly equivalent to C using "static globals" instead of named local variables, right?

there are some important differences

i just edited in a note about one of them, the scoping

for large programs forth has namespaces (vocabularies or wordlists) similar to static file scope in c

an important similarity between standard forth variables and c static variables is that they don't support recursion. in the occasional case where you do need recursion you can use the operand stack to save and restore your variables manually

    x @ y @ ... recurse ...
    y ! x !

which would not be possible in c with universal use of static global variables

also though typically when you are stepping through forth you are doing it by typing names of words, while in c you use the debugger, which can see and change local variables

Seems to me more like declaring variables in C as static, inside function definitions. The names might be duplicated, but inside that function it will always point to the same common memory location.

yes, it's similar in that every function can have its own variable x if you want, but an important difference is that you can share the same variable between several forth words, as in my example in https://news.ycombinator.com/item?id=33786716, even if you also have other variables of the same name used by other groups of words

this allows you to make your words more like 'named lines of code' or 'named parameters' than like c functions, which is helpful for interactively poking at the system, and that's where forth really shines

like historically, as i understand it, the ui to the forth, inc. fullscreen text editor was a forth repl, just using a vocabulary with a lot of one-letter words for editor commands. that's not practical even with lisp; imagine trying to use emacs with only m-: as your keyboard interface. but with forth the interaction friction is only roughly as bad as with ed or ex, with the bonus that you can see what you're doing

in theory making your words smaller helps with composability as well, but having hidden state shared between the words in variables does tend to undermine that benefit imho

At FORTH, Inc. in the 80s they programmed the keyboard to make the arrow keys (and such) macro-expand into editor-vocabulary commands. So the Forth-repl editor had interactivity sort of halfway between ed and Emacs.

(Related anecdote: my summer job there back then was to review the new version of their system to find bugs, etc. There was this one piece which was ugly and awkwardly coded, and I was kinda happy to find it: finally, an excuse to rewrite something! It was the conventional interactive editor. Turned out it had been contributed by a customer; absolutely nobody at the company used it. Forth editor all the way.)

thank you for the correction; i was relaying my poorly-remembered account of what you said about the experience, as you likely guessed

Did you ever read "Thinking Forth"?

a few times, mostly last millennium

In that case, your Forth style is fascinating. Cheers! :)

don't get me wrong, i do think

    : square dup * ;

is better code

but the possibility of such clean, concise definitions is a seductive trap; it is easy to write code that isn't obviously correct in pursuit of that aim

quickly this leads you to code that is not just unreadable but full of bugs, and that is how the person who started this thread was, in their words, trying forth and failing; this is not just a common experience but nowadays a nearly universal one

so it is much better to err on the side of too many variables and too little stack manipulation than the opposite

you can always streamline the code later once it works

i'm pretty sure leo brodie would agree with me on this

I was just curious, you seem like a meticulous thinker so I figured you had done your homework before adopting an unusual Forth style. Cheers

oh, apologies for misinterpreting your comment as meaning 'too bad you didn't understand the book any of the times you read it'

my defensive tone was uncalled for

thank you for making the additional effort to clarify despite it

coincidentally i gave away my paper copy last night, six days after your comment and, as it turns out, 25 years after buying it

I tried to write a simple program in Forth that needed to keep track of a list of pairs of numbers and after coming up against many issues I asked online in a Forth group and was told that Forth simply isn't suited for such programs (programs that need to work with dynamically sized amounts of arbitrary data). Maybe this is also true for your case

that is maybe an exaggeration but forth is certainly more designed for reactive systems than transformational ones

video games and motor controllers rather than computer algebra systems

the difficulties are the same as in c: you have to do your reallocation and deallocation and probably hashing manually

a surprising number of such problems turn out to work well enough if 'dynamically sized' becomes '1 billion or less' which simplifies the memory management dramatically

and you can write theorem provers and optimizing compilers and stuff in forth; it just isn't its strong point

I believe there are two big problems with Forth

1) losing track of the parameters in the stack (unlike Lisp, there are no parens to match and serve as containers)

2) everything is a number/pointer. STOIC provides strings, hash tables and more, along with parameter type checking.

Those are, in my opinion, the two main factors that keep Forth from being the "dual" to Lisp.

these are real annoyances but most people get stuck in the stack manipulation tarpit first before those become the main problems

2over should be safe, legal, and rare

Well, there is Kitten, although it hasn't seen an update in two years and was moving quite slowly before that too.

https://kittenlang.org/

yeah, cat, kitten, and joy are interesting. sadly factor is abandonedish too because slava is too busy cloning kotlin

Nah, johnb, erg and others are putting in a lot of effort. Join the Discord if you want to keep up to date, QxJYZx3QDf.

sweet, thanks, but i don't saas much

Forth is extensible, it is not complete out of the box like a C compiler. Just extend it with named local variables and use them. There is no shame and unless you are on some rare and exotic stack machine, no performance penalty using local variables. I also need to add safe strings with their own memory pool, rather than block copying bytes around in the same dictionary where my code lives. I am just not a good enough programmer to use Forth strings without crashing a lot.

from my non-expert viewpoint, the big penalty for named local variables in forth is that it is a barrier to breaking up a word into smaller words, because pieces of code inside the same word that communicate through a local variable will stop working if you break them into separate words

bigger words impede using the forth repl as a line-by-line stepping debugger, so they diminish the interactivity that is the reason you'd use want to forth in the first place

agreed about strings; one of the first things i wrote in forth was a straight port of bernstein's stralloc, last millennium

If you can keep variables separate between different words, you can make sure that the variable stays local to the word. It is error-prone but can stop the variable from leaking and keep words separate.

what do you mean

Define variables before you start defining words. Make sure you use one variable per word. It's a manual process but when composing words helps make sure variable collisions don't happen.

you don't need to do that, you can define a separate variable named x above every colon definition if you want

it hides the earlier declarations of the same name

Sure, I prefer not shading but that works too.

> 1) losing track of the parameters in the stack (unlike Lisp, there are no parens to match and serve as containers)

If you're losing parameters in the stack, you need to refactor. The whole idea of wrapping things in brackets is what makes Lisp such an unreadable mess.

> 2) everything is a number/pointer.

If you don't understand why everything is a number or pointer, you don't understand anything about programming languages.

It is all number or pointer all the way down but that's not the best interface to interact the computer with.

It depends what you're doing.

What would be a better interface for dealing directly with something like IO ports?

Kragen, how much of what you said here about Forth also applies to PostScript, do you think?

I have been tinkering with PostScript but not Forth, I grok that PostScript is a lot closer to LISP. A lot of people (without actually learning these languages/systems) say Forth and PS are similar, but I know that's not really the case - their resemblance is fairly superficial.

i agree, and though it is certainly possible to get ensnared in dup exch baling wire in ps, it isn't nearly as common, maybe because you can just 6 dict begin for some tasty recursion-safe local variables when you have the appetite

or maybe because you can't >r r@ r@ rdrop, or because garbage-collected [arrays] remove the temptation to write subroutines with 6 parameters

ps is fairly big compared to forth, not the 16 megs of libgs.so.9.50 but definitely a lot more than 8k

what they have in common is interactive repl orientation, user-defined control structures, great flexibility, lack of static types, and of course rpn

i was going to say 'closures' but of course ps's scoping is dynamic so it lacks them

I once wrote a robot controller in Forth, because that was the only thing available on a dinky microcontroller in the mid-1980s.

Would not have done it if the C compiler for that target had been cheaper. We have better tools now. Back then, it was one of the few ways to do interactive development on really tiny machines. In that niche, it was useful.

It's been around for decades and has virtually zero traction, which is to say, Stop trying to make Forth happen.

i mean, you're not wrong, walter, you're just

it's unlikely to displace c or rust or python or unix or freertos or openocd

but it does have something special, like a delorean or something; unlikely to be forgotten either

Not all programming is about making maximal products in minimal time. I also use hand tools all the time, a lot more than if I was a carpenter billing by the hour. How artless one must be to worry about 'traction' off the clock!

there is

one art

no more

no less

to do

all things

with art-

-lessness

(piet hein, grooks)

Reply on news.ycombinator.com