lispm

You are on the surface. I'm talking about a computing paradigm defined by LISP. A foundational model at the core of the language, enabling us to understand how the List Processor actually works.

LISP has dialects which are not homoiconic, where programs are not written as s-expressions, ... But the core is the same: a specific List Processor. McCarthy himself did not want to write programs as s-expressions. He defined M-Expressions, where S-Expressions were only used for the data.

Check the book "Anatomy of LISP", which is a classic. All the code in the book is written in M-Expressions. Now, Racket gets a new syntax (-> Rhombus). But the language core mechanisms are still there. I'm a fan of the s-expression syntax, but I've also heard&read from many people (incl. McCarthy), that THEY preferred a different syntax. I've seen Apple defining Dylan (Dynamic Language) with an s-expression syntax and then changing the language to a different syntax for broader adoption. You (and me) think it is essential. McCarthy and others thought that it was the wrong syntax for getting wider adoption.

Btw., personally, I don't care that much what Rich Hickey says. I can think for myself.

LISP does not stand for "List processing", it stands for "List Processor". To quote John McCarthy: "A programming system called LISP (for LISt Processor) has been developed for the IBM 704...".

This is a programming system, which is a list processor. Not a vector processor, not a graphics processor, not a number cruncher -> it's a list processor.

Now McCarthy defines three basic LISP features:

* Data has the form of S-expressions, for which a definition is given, which is linked lists of atoms&lists. -> a very low level definition.

* There is a LISP language which processes these s-expressions. Simple operators are being defined.

* LISP can interpret and execute LISP programs in the form of s-Expressions.

To sum up: there are linked lists, a programming language for processing them AND the programs themselves can be interpreted by LISP.

He then defines an "universal LISP function", an interpreter for LISP in LISP.

That's the actual core essence of LISP. Think about it!

That's the big thing which was defined in the very first pages of the first LISP papers: a very simple foundation, which describes how to do computation by a List Processor, applied to itself.

This is the core language: a data structure, a few operations and an interpreter written in itself.

Fire up your Emacs Lisp (start GNU Emacs and do m-x ielm). It's all there: the data structure (-> linked lists of symbols), the operations (car, cdr, cons, read, print, cond, lambda, eq, assoc, append, ...) and the interpreter (-> eval).

It's not one of many data structures, it is the central thing: s-expressions, programs as s-expressions, a list processor (-> EVAL) defined in the language it is executing, executing programs which are given as s-expressions, able to execute itself.

All the other stuff you've mentioned is secondary: Homoiconicity, First-class Functions, Macro System, automated memory management, REPL-driven, interactive development.

McCarthy described a specific way to do computation. That's the core of LISP. One which can be taught in a day, which fits on a few pages and which allows us to develop a simple mental model of it.

Clojure does not have the same axiomatic definition of a List Processor, it is not interpreting lists, ... It has a complex data structure at its core: lazy persistent sequences, it compiles code (the compiler is written in Java and compiles to the Java Virtual Machine), it has the JVM underneath, ...

If I break the execution of a list processor, I see the programs as lists being executed by a lisp processor. Write a program in Emacs Lisp, execute it by the interpreted EVAL, run it, interrupt it and you'll see a backtrace where the interpreter runs the Lisp expressions.

If I want to teach people Lisp, I explain this simple LISP PROCESSOR. That's the mental model they need to have: the form of Lisp programs and its execution. Once they have this mental model of LISP, it is valid for all core LISP dialects.

This is also explained in the LISP literature: "LISP 1.5 manual", "Anatomy of LISP" (John Allen -> the book is available here for download: https://dl.acm.org/doi/10.5555/542865 ), "Structure and Interpretation of Computer Programs" (Abelson/Sussman), "Paradigms of AI Programming" (Norvig), "Lisp In Small Pieces" (Queinnec), "The roots of LISP" (Graham) and others. All these explain this foundation (and expand on it).

Clojure is slightly different. It's a modern&pragmatic&opinionated dialect of Lisp for the JVM, which does not have this simple LIST PROCESSOR. That's why it is an dialect of LISP, but not one which shares the axiomatic core of LISP.

one can easily miss the linked examples, here are the links from the GTFL page:

http://www.martin-loetzsch.de/gtfl/application-example-1/

http://www.martin-loetzsch.de/gtfl/application-example-2/

But why is a list not a list across all platforms?

Other Lisp dialects run on top of the JVM, .net, JavaScript, LLVM, UNIX, iOS, Android, Linux, ... and there is no such problem.

Try ABCL on the JVM. (listp (cons 1 '(2 3))) is T.

I would think that consistency and correctness of basic Lisp operations (-> LISP stands for "List Processor") would be a high priority of any self-respecting Lisp dialect?

Your example looks like Clojurescript? It is also inconsistent between variants?

  Clojure 1.11.3
  user=> (list? (cons 1 '(2 3)))
  false

and then you get this in Clojure:

  user=> (list? (cons 1 '(2 3)))
  false
  user=> (cons 1 '(2 3))
  (1 2 3)

That's "strange", isn't it? It prints as (1 2 3), but it is not a list? So there are things which print as lists, but aren't lists? What? Which also means that when we print it and read it back, it will be of a different type and list? will be true?!

  ELISP> (listp (cons 1 '(2 3)))
  t

Puh, Emacs Lisp got it right.

I usually don't want to call people "idiots", but I would want to prevent confusion. Even though English is a Germanic language, it's not German. It's a matter of expectation. If a book title says "Lerne Deutsch" it's not meant to mean "Learn English". Same for "Learn Lisp" and "Learn Clojure". Both books usually will be about different languages. For example there was a book "The Little Lisper", for Scheme it was renamed the "The Little Schemer". People then knew that the book is using the Scheme dialect of Lisp and not Lisp itself.

Btw., even though "lernen" and "learn" are coming from a common language background (Proto German, also called Common Germanic, no joke -> https://en.wikipedia.org/wiki/Proto-Germanic_language) and here mean the same.

"continuations" are not in standard Common Lisp. They are a Scheme feature (-> functions which capture a state of computation). Common Lisp has a standardized exception handling system called the "Condition system", it features conditions (-> exception classes), handlers and restarts. It provides a non-terminating exception handling system, also often with a user interface -> the user can choose a restarts via some menu system or in the REPL. Something similar is in Smalltalk, which provides excellent tools for error handling and debugging.

Lisps prior to Common Lisp usually had elaborate ways to handle runtime errors and extensive REPL interfaces for it, for example the break loop -> a REPL in the error context. The most prominent and widely cited was the one of Interlisp, which featured also user facing automatic error repair (-> DWIM).

Whenever I see a Lisp, which doesn't stop at an error and then let me look at it, I feel sad.

GNU Emacs is an application (a text editor) -> debug-on-error is by default NIL. That makes sense in an application.

> that never became a thing everyone wants, I guess it's not as critical as you paint it to be.

People asked for it, it seems. But the answer was no.

See for example "https://vimeo.com/223309989", "REPL driven development". It was a topic in the Q&A at the end.

Also: Since JVM/Java already has a built-in exception handling, why build another one?

There are some libraries...

"Being a Lisp" or "being a dialect of Lisp", these are two different things for me.

But even the "dialect of Lisp" actually is more like "spiritual dialect of Lisp".

Clojure was from the beginning designed with exactly ZERO backwards compatibility to Lisp. Exactly ZERO of the existing Lisp programs/libraries could work. It was even difficult to port them. It's basically a kind of "dialect" with new operators and data structures, which interprets prior concepts in new ways and mixes in new stuff.

But it was designed with compatibility to Java and the JVM -> one could not reuse existing Lisp code, but one could reuse existing Java code -> since Clojure was hosted on the Java and had integrated calling Java code. Reusing Java was more important, because the goal was to work in a Java industry. Thus there wasn't a version of Clojure on top of Lisp, only versions on top of languages/infrastructures like JavaScript or .net, because there was another source of reuse.

If we look at the above linked paper "The Evolution of Lisp", then "Evolution" actually meant "Evolution", not "Restart" -> the language evolved by improving, adding, extending, removing, ... around a core design.

Clojure was designed as a new dialect, free of the baggage from the past, with a completely fresh start. Reusing existing code or evolving an existing language was a non-goal. It was newly designed as a hosted implementation (-> the Lisp runtime was gone), it was designed with lazy data structures at its core (-> the Lisp linked lists were gone), it was an opinioned new design, based upon of what Rich Hickey liked about Lisp and what would help him to be employed as a consultant in a Java world -> by developing upon and within the Java stack of standards, tools and libraries.

And that's fine for him. Rich Hickey had his goals and succeeded with them. Other people found it useful, too.

Common Lisp and/or its implementations (and a bunch of other dialects) have most most of the features of the original LISP directly, often with minor changes, like many of its operators and data structures (like the linked list). It was designed to be largely backwards compatible with earlier LISP. Original LISP implementation features like the self hosting machine code compiler, loading of natively compiled code and the ability to save/restore images of the Lisp heap (which really was already in the 1960s LISP) are also provided by various later Lisp implementations, incl. several CL implementations.

Uncut version: https://www.dreamsongs.com/Files/HOPL2-Uncut.pdf

A Pattern of Language Evolution, Gabriel/Steele 2008, https://www.dreamsongs.com/Files/PatternOfLanguageEvolution....

On the positive side, one can also study the works of brilliant minds... like the original work on the Common Lisp Object System and its metaobject protocol. It added support for object-oriented programming to Lisp and was originally developed with a prototype implementation, which could be loaded into a running Common Lisp.

https://mitpress.mit.edu/9780262610742/the-art-of-the-metaob...

The toy implementation from the book: https://github.com/binghe/closette

A real implementation, here for SBCL: https://github.com/sbcl/sbcl/tree/master/src/pcl

> CL is like clay,not even lego bricks.

There is also the "Big ball of mud" metaphor, which is attributed to Joel Moses, comparing it to the diamond like structure of APL:

  "Lisp, on the other hand, is a ball of mud.  It looks ugly, but you
   can add stuff to it without changing its character."

"self tail recursion" is only a simple case of the idea of tail calls (-> tail call optimization, TCO). In a Lisp variant/implementation with TCO (generally Scheme implementations and also a bunch of CL compilers, like SBCL), ANY tail call is automatically optimized to a jump, without growing the stacks, without the need to use special construct like Clojure's loops or trampolines.

for example SBCL on ARM64

    CL-USER> (defun example (a b)
               (if (> a b) (sin a) (cos b)))
    EXAMPLE

In above CL function, the calls to SIN and COS are tail calls. Let's look at the machine code, which is automatically and by default created by above definition:

    CL-USER> (disassemble #'example)
    ; disassembly for EXAMPLE
    ; Size: 100 bytes. Origin: #x7006948100                       ; EXAMPLE
    ; 00:       AA0A40F9         LDR R0, [THREAD, #16]            ; binding-stack-pointer
    ; 04:       4A0B00F9         STR R0, [CFP, #16]
    ; 08:       7A0300F9         STR CFP, [CSP]
    ; 0C:       EA030EAA         MOV R0, R4
    ; 10:       EB030DAA         MOV R1, R3
    ; 14:       FA031BAA         MOV CFP, CSP
    ; 18:       29BE80D2         MOVZ TMP, #1521
    ; 1C:       BE6B69F8         LDR LR, [NULL, TMP]              ; SB-KERNEL:TWO-ARG->
    ; 20:       C0033FD6         BLR LR
    ; 24:       4EB741A9         LDP R4, R3, [CFP, #24]
    ; 28:       5F011DEB         CMP R0, NULL
    ; 2C:       E1000054         BNE L0
    ; 30:       EA030DAA         MOV R0, R3
    ; 34:       F6FBFF58         LDR LEXENV, #x70069480B0         ; #<SB-KERNEL:FDEFN COS>
    ; 38:       570080D2         MOVZ NARGS, #2
    ; 3C:       DE9240F8         LDR LR, [LEXENV, #9]
    ; 40:       DE130091         ADD LR, LR, #4
    ; 44:       C0031FD6         BR LR
    ; 48: L0:   EA030EAA         MOV R0, R4
    ; 4C:       76FBFF58         LDR LEXENV, #x70069480B8         ; #<SB-KERNEL:FDEFN SIN>
    ; 50:       570080D2         MOVZ NARGS, #2
    ; 54:       DE9240F8         LDR LR, [LEXENV, #9]
    ; 58:       DE130091         ADD LR, LR, #4
    ; 5C:       C0031FD6         BR LR
    ; 60:       E00120D4         BRK #15                          ; Invalid argument count trap

One can see in the assembler, that the > function is called as a subroutine with BLR (-> it is not a tail call), while SIN and COS are called with a BR, basically a jump, since those are tail calls. Thus in any piece of code, the compiler recognizes the tail calls and will create the corresponding machine code jmp instruction (minus any restriction a compiler will further have).

Generally Common Lisp doesn't require TCO, though many compilers provide it as an optimization. Required is a basic low-level lexical goto construct, which is used to implement various loop.

> but GraalVM really does help with that

In something like SBCL there is no special tool/implementation (restricting dynamism) needed to create an executable. It is a function call and one second execution time away. The resulting executable starts fast and is basically a full Lisp with all features, again.

> More a curiosity than something practical and it's looking like it still belongs in the 70s.

Emacs also belongs to the 70s. The UI basics were from TECO Emacs, mid 70s and many are still there. Stuff like the "Meta" key which no longer exists, or the buffer metaphor. EINE/ZWEI and Multics Emacs were also developed then. Again, the basic stuff is still in GNU Emacs. Cursor handling in GNU Emacs is like it was in the 70s.

> For all intents and purposes, Emacs is the 'best' freely available and most used Lisp machine we have today

I don't see GNU Emacs as a "Lisp Machine", it's a mostly text-ui based development platform & environment written in C+Lisp and its usage is at the same time limited by its oldish user interface. There is a lot of useful software written in it, but the actual broader use of GNU Emacs is limited to a very nerd-like group. I've used GNU Emacs to read/write mail (for example). But that was long ago. There is a limited appeal of the text-oriented UI, which also has a hard time to adapt to devices like phones or tablets.

> practicality of Emacs

The practicality of GNU Emacs is also limited, because its user interface would not reach larger groups of current users.

> but I barely touch it these days, as I tend to reach for Emacs Lisp instead

It's also funny that many people would like to move GNU Emacs Lisp more towards Common Lisp, but they are not allowed to do so... -> see the repeated discussions (since many many years) on the developer mailing list...

> This level of introspection goes beyond anything available today except some Smalltalk environments (which lose on practicality).

GNU Emacs has a bunch of user-facing C code which can't be changed live.

Many Lisp systems (and also applications) provide runtime introspection&reflection&modification. See for example Interlisp ( https://interlisp.org ) which was already a full live programming environment with managed source code in the 70s.

> Lisp syntax doesn't just feature lists, Lisp syntax is those lists (Lisp = LISt Processing)

Lisp syntax is on top of lists. Lisp has as basic forms functions, built-in operators and macros. Each new macro implements syntax.

> On the flip side of this, how one is meant to distribute finished Lisp programs is very unclear to me.

In SBCL you would compile all the code to machine code. A program is then a single file with the Lisp runtime (providing threads, memory, garbage collection, OS, ...) and a snapshot of the Lisp heap (all the Lisp objects in memory), which is called a Lisp image. That's an executable. If you start the executable, it starts the Lisp runtime, restores the heap from the file and then hands over control to a defined entry point in the heap.

ECL (another Common Lisp implementation), compiles all CL code to C and then uses a C compiler to create loadable compiled C code. The runtime then loads the compiled code on startup.

LispWorks does something similar like SBCL, but LispWorks can remove unneeded functionality from the heap plus can optimize the needed space. Thus the applications can be more static and also can be smaller. Imagine one would write a, say, CAD application in LispWorks. It could start into a GUI application and the user may never see any Lisp REPL. But a REPL could be a tool for the user, where she/he can script the CAD system.

The "standard" has a huge amount of undefined behavior and also defined unspecified behavior. For example for everything "system class" it is undefined how this thing is implemented - there is an API specified. An example is the condition system: it is not required that a condition is a CLOS object. There is also a bunch of things which can be implemented or ignored. It is fully standards conforming to ignore all type declarations. About some central Lisp features, there is nothing in the standard. Garbage Collection is such a thing. The standard does not even mention it, thus it is also not requiring it. It says nothing about invoking a GC or an interface to the GC.

It's also not forbidden to define subsets of the standard. For example there were several implementations of CL to C compilers developed, which lack a lot of dynamic behavior at runtime. L is a subset for embedded systems. SubL is a variant used to program Cyc. CLICC was used to compile the applications of the expert system shell Babylon to C.

Then there are implementations with large sets of extensions to the standard: streams in CLOS, conditions in CLOS, hash tables with various extensions, various extensions to TRACE and STEP, MOP for CLOS, ...

So there is a range from a primitive CL (implementing only the minimum requirements) to a fully CLOS-based CL with lots of extensions. In between are exotic & large implementations like Genera, where parts are written in Flavors, for example there are several dozen PATHNAME (for UNIX, Mac, DOS, VMS, ... pathnames) classes, which are implemented in Flavors and where (pathname ...) has parsers for various filesystems. SBCL devs have also experimented with extensions like extensible sequences.

It's not that difficult. Lisp makes it easier, because one can look at code.

For example look at this built-in Lisp macro example, which iterates from 0 to 9 and writes the numbers.

    CL-USER> (dotimes (i 10)
               (write i))                                                                                                                                                               
    0123456789             
    NIL

Common Lisp has tools like a pretty printer for code (-> PPRINT) and a tool for expanding a macro form:

    CL-USER> (let ((*print-right-margin* 30))
               (pprint (macroexpand '(dotimes (i 10) (write i)))))                                                                                                                                    
                           
    (BLOCK NIL                                                                                                                                                                                        
      (LET ((I 0))         
        (DECLARE                                                                                                                                                                                      
         (TYPE UNSIGNED-BYTE I))
        (DECLARE (IGNORABLE I))                                                                                                                                                                       
        (TAGBODY           
          (GO #:G267)                                                                                                                                                                                 
         #:G266            
          (TAGBODY (WRITE I))                                                                                                                                                                         
          (PSETQ I (1+ I)) 
         #:G267                                                                                                                                                                                       
          (UNLESS (>= I 10)
            (GO #:G266))
          (RETURN-FROM NIL
            (PROGN NIL)))))

From above one can see that the code gets transformed in very low-level Lisp code, with a GO operator (-> a GoTo). It's a low-level loop expressed by loop made from a simple goto operator. One can look at the code and try to figure out what it is doing. One is not guessing what the expanded code looks like, one can generate the expanded code and see what it looks like.

It's not that misleading, though, true, both provide various interactive development environments, with Smalltalk versions very much GUI tool based (with tools like the System Browser).

Typically when one starts a Lisp development system, it will always start some kind of visible REPL. Either it runs it in a Terminal (most UNIX Lisps have a terminal REPL), runs it as a tool in its own GUI-based IDE (MCL, CCL, Allegro CL, LispWorks, Corman Lisp, ..., Symbolics Genera, CL + McCLIM) or runs the REPL frontend in a connected IDE (-> SLIME + GNU Emacs is a prominent IDE for CL, earlier this was done using subprocesses called "inferior Lisps" like in ILISP+GNU Emacs).

The conversational REPL interface (typing expressions, evaluating the expression and getting a response) is widely used.

Smalltalk has the "Workspace" or a Playground (-> Pharo) as similar tools.

If we look at macros in Scheme R7RS they are mostly transformers on patterns. -> https://standards.scheme.org/corrected-r7rs/r7rs-Z-H-6.html#...

> If your not careful to use facilities like gensym and packages it easy to accidentally capture already binded terms at call site.

Luckily CL programmers have heard about that and are usually careful about that.

The language features that make this possible are a) late-binding and b) resident development tools (compiler, debugger, interpreter, inspector, trace, ... are all included in the runtime).

The REPL often is also the default debugger. On error one gets a debug REPL at the point of the error (no unwinding), one debug level deeper. The debug REPL is basically a normal REPL, but with some debug features (stack movement, ...) and the ability to call predefined restarts in the code.

> compiled language

more like "batch compiled language". In many Lisps one would incrementally compile code. Common Lisp has already three functions built-in: LOAD, COMPILE and COMPILE-FILE. How they work is depending on the implementation, but LOAD typically can load source and compiled files, COMPILE compiles a single function in memory, and COMPILE-FILE compiles a source file to a compiled file (native machine code or some byte-code). SBCL for example compiles with these functions every thing to machine code, which it does anyway, by default.

One of the things which make this useable for incremental work is the symbol table, which holds all symbols and its values&functions. One can change the function of a symbol at runtime and global function calls go through this symbol table (-> late binding). So an update to a function has effect on its next call.

what can go wrong?

In Lisp one could have (-> a b c d), just like (+ (+ 2 3) 4) is (+ 2 3 4).

I wasn't responding to your comment.

I wonder how much performance will be used to run all the advanced Microsoft telemetry software (which collects & reports user data for Microsoft and which can't be disabled) and all that ad software they force on users.

I still wonder how it is legal to use Microsoft software in the EU in government surroundings. The German authorities publish tons of papers on how to use Windows&related ( https://www.bsi.bund.de/SiteGlobals/Forms/Suche/EN/Servicesu... ) , instead of simply disallowing to use Microsoft software.

With the data collection for "AI" it only gets worse.

Macs were long hindered by Intel processors to have long battery lifes AND/OR good performance. Two examples of MacBooks I had:

The 12" MacBook. Great form factor, bad software (graphics driver code was especially poor), poor keyboard, lousy battery life, slow CPU & graphics. Similar size now with the 13" MacBook Air, but much better.

The i9 MacBook Pro. Too heavy, not enough cooling, extra GPU chip (in the laptop) with lots of power demand, Thunderbolt chips with lots of power demand, CPU with lots of power demand, fans were often on (for example with anything which uses the extra GPU). Now a 16" MacBook Pro with Apple Silicon is virtually silent AND fast.

I currently have a Mac mini with M2 Pro, a MacBook Pro with M2 Pro and the new iPad Pro M4. All are very snappy and virtually silent. They use very little idle power and don't get warm in normal use. The i9 MacBook Pro would produce a lot of heat when attached to an external screen and doing a "simple" video call, due to the power hungry additional GPU needed to drive the external screen.

There are a bunch of confusions to the relationship of LISP, the LISP system and the LISP paper of 1960.

The "original paper on Lisp" is from 1960: http://www-formal.stanford.edu/jmc/recursive.html

A bunch of papers about the design and development of LISP and the LISP system had been published earlier. Those show the various stages of the project to define and implement the new language. See for example: https://www.softwarepreservation.org/projects/LISP/lisp15_fa...

By that time of publishing the paper on LISP, an actual "LISP system", the practical implementation of LISP existed. To quote from above's paper: "A programming system called LISP (for LISt Processor) has been developed for the IBM 704 computer by the Artificial Intelligence group at M.I.T.". "has been" and not "will be".

The manual of this first LISP system, called Lisp I, is here in a version of March 1960: http://bitsavers.org/pdf/mit/rle_lisp/LISP_I_Programmers_Man...

The "LISP Programming System" is described in Chapter 4. It had functions, numbers (floats) and a machine code compiler. Chapter 5 describes input from punched cards, function tracing and a interface to an online printer/keyboard. Chapter 6.3 describes the garbage collector. Chapter 9 describes the available (ninety) functions, including eval, map, maplist, print, read, compile, matrixmultiply and a bunch of others.

So it is fair to say that the development of a theoretical foundation and the LISP system itself was not just "somewhat interleaved", but were basically completely interleaved, both together were done incrementally.

The process was not to 'first define theoretical foundations, then a language definition and then an implementation'. It was always all three combined, with a real practical LISP system on a real computer, developed incrementally at that time.