lispm

> Pushing items onto a stack isn't inherently a prepend operation.

a stack has no idea of "prepend" or similar. It's defined by abstract like operations PUSH, POP, IS-EMPTY, MAKE-STACK. How it is implemented is an implementation detail.

> It's generally a "put it on an end" operation,

Which you made up. There is nothing "generally" like that.

> meaning you could append it. Now, with a linked list that would be slower,

and to append would be unnecessary.

> So if I do implement a simple stack, I implement it with a vector (example[1]).

That's a simple "bounded" & "fixed size" stack.

> I'm a professional.

Okay. ;)

> Yes, I know the answer you're fishing for is "immutability"

That would be one answer. Can you think of more?

> So, you did tests to show that linked lists are slower AND use more memory? Which we both already knew? Why?

I showed you that it often does not matter much.

> Oh good, I'll keep that in mind for the next time I am forced to prepend a bunch of items to a list, which I expect to happen approximately 0 times before I die of old age.

It's not that uncommon, for example for a simple stack implementation.

> So... what I'm hearing here is, that no, you aren't capable of admitting that Lisp has faults.

With this discussion style you won't get very far.

> It seems to me that being slower and using more memory and getting nothing in return isn't a tradeoff.

Right, I think that's a useful task for you to actually think about: what do we get in return?

Why does Haskell use linked lists?

Why does INRIA's OCAML ( https://ocamlbook.org/lists-and-structural-recursion/ ) ?

Why does Microsoft's F# ( https://learn.microsoft.com/en-us/dotnet/fsharp/language-ref... ) ?

Why does Ericsson's Erlang ( https://www.erlang.org/doc/system/listhandling.html ) ?

Kind of. A lot of the GC support in the Symbolics 3600 architecture is on the Microcode level (current CPUs usually don't have such operations in Microcode). The word size of the CPU is 36bit. The CPU operations don't deal with the type and gc tags on the instruction level, this is done on the Microcode level. Things like "invisible pointers" are also dealt on the Microcode level.

Ephemeral GC, concentrates on garbage collecting objects in RAM: For example every memory page has a page tag, which marks it modified or not. The ephemeral GC uses this to scan only over changed pages in memory. The virtual memory subsystem keeps a table of swapped-out pages pointing to ephemeral pages. The EGC can then use this information...

> "Where necessary?" is almost always, because vectors almost always outperform linked lists based on cons cells.

Let's see, I have here a Common Lisp compiling to machine code, here non-optimized generic Lisp code:

I'm allocating n elements list/vector of vectors and reverse it p times. reversing allocates a new list/vector.

    (defun test1 (n p)
      (let ((v (make-array n)))
        (map-into v
                  (lambda ()
                    (make-array 10 :initial-element 1000)))
        (loop repeat p
                do (setf v (reverse v)))
        v))

    (defun test2 (n p)
      (let ((l (make-list n)))
        (map-into l
                  (lambda ()
                    (make-array 10 :initial-element 1000)))
        (loop repeat p
                do (setf l (reverse l)))
        l))

    CL-USER 15 > (time (test1 10000 100))
    Timing the evaluation of (TEST1 10000 100)

    User time    =        0.005
    System time  =        0.000
    Elapsed time =        0.003
    Allocation   = 9088696 bytes
    1 Page faults
    GC time      =        0.000
    #(# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ...)

    CL-USER 16 > (time (test2 10000 100))
    Timing the evaluation of (TEST2 10000 100)

    User time    =        0.008
    System time  =        0.001
    Elapsed time =        0.006
    Allocation   = 17113288 bytes
    842 Page faults
    GC time      =        0.003
    (# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ...)

The list or vector is 10000 elements long and 100 times reversed and the difference in runtime is tiny. The list version is only marginally slower, but allocates a lot more memory.

I can also construct cases, where the list is much faster. Adding a one element list/vector to the front:

    CL-USER 22 > (time (test1 10000 100))
    Timing the evaluation of (TEST1 10000 100)

    User time    =        0.016
    System time  =        0.000
    Elapsed time =        0.012
    Allocation   = 9148776 bytes
    0 Page faults
    GC time      =        0.000
    #(A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A ...)

    CL-USER 26 > (time (test2 10000 100))
    Timing the evaluation of (TEST2 10000 100)

    User time    =        0.000
    System time  =        0.000
    Elapsed time =        0.000
    Allocation   = 1042248 bytes
    0 Page faults
    GC time      =        0.000
    (A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A ...)

Here the runtime for lists is much faster, since it does not copy the complete list or -> lists share storage, consing to the front is cheap.

-> for many practical use cases for lists the total runtime and the runtime difference does not matter

Obviously there are also many case where a list would perform much worse, like accessing the last element of a list.

> Are you even capable of admitting that Lisp has serious faults

You seem to be keen to find "faults". Usually things are not black and white. There is no true and false, but engineering trade-offs. These tradeoffs can Lisp make the wrong choice for problems. Many of the trade-offs are also influenced by non-technical issues: like the amount of engineering time invested into a language implementation: see SUN/Oracles investment into Java.

> My critique was about list implementation, not about implementing the entire runtime.

Linked lists are actually managed by the runtime.

Example on a esoteric platform, a Lisp Machine, a real, but outdated, computer. But we still have Virtual Lisp Machines, which have a virtualized implementation (not as in a microcoded CPU, as the real ones were).

If I cons a list via CONS, I get a linked list made of cons cells. But over the lifetime of the list, this may change. At some point in time the runtime may decide to copy the list (for example because it has a copying garbage collector). The effect then is that the list is allocated in contiguous memory. The program itself does not see a difference.

First the list (1 2 3) is allocated as [1 | -> [2 | -> [3 | NIL]]], written in Lisp as (1 . (2 . (3 . NIL))) . At some point in time the runtime changes this transparently to [1][2][3], where the cells are tagged, such that the successor is in the next memory word and where the last element is tagged that it has no successor. That's called cdr-coding.

Now If I copy a list, with COPY-LIST I always get a cdr-coded list as a result. Several list functions return cdr-coded lists, instead of linked lists.

This cdr-coding method is not much used anymore, since there is not much of an advantage: either one uses linked lists (because they are easy to use and have useful properties, like that CONS does not need to copy its arguments) or other available data structures (vectors, structures, ...).

What is still used are locality improving garbage collectors.

> Using garbage collection is not equivalent to using the entire Lisp model. Java, JavaScript, Python, Lua, Ruby, C#, etc. do not make extensive use of linked lists in their standard libraries, instead preferring--you guessed it--vectors.

Lisp also makes use of vectors, where necessary. These languages with managed memory like Java have the same object/reference model as Lisp and the same calling mechanisms. linked lists are only a special case (-> java.util.LinkedList).

Say we have N cities. Each city is a an object of class CITY. Each city will have attributes, for example NAME and PEOPLE-NUMBER. Now we want to keep two "lists" of them one sorted by name and another one sorted by PEOPLE-NUMBER.

We need two vectors and N city objects. In Lisp the city objects are not stored in the vectors. They are stored somewhere in the heap. -> there goes your "contiguous memory" to bust. The vectors are pointers into a heap. Every access to the nth city object through one of the nicely contiguous vectors, references an object which is stored somewhere else.

That model is used in many languages implementations. It's the Lisp model of memory management, introduced with Lisp 1 -> dynamic memory management plus a garbage collector to reclaim unused space.

> And if you dig into how structs are implemented in say, C#, they are implemented as contiguous blocks of memory.

That's also the case in Lisp. But a struct (or list or vector) of structs usually points to those being somewhere on the heap. A struct/list/vector of structs is not a single contiguous block of memory. structs in slots are typically not inlined. Some languages inline data non-primitive structures, Lisp usually does not.

> contiguous blocks of memory

That's can be an illusion. In a virtual memory system, the memory is made from pages of a fixed size. Random pages are cached in RAM, influenced by their usage pattern over time.

> in most languages it does not entail pointer chasing

It does, see above. Just not tail pointers in the list.

> Ironically, higher order functions like map/reduce/etc. push you toward operating on lists as a whole which is exactly the case where vectors most outshine linked lists.

That's why in Common Lisp the functions MAP and REDUCE work over vectors, too.

> Lisp-family languages are fairly unique in this problem.

see Prolog and various functional languages, ... some even have basic strings implemented as linked lists of characters (which Common Lisp does not do).

If we look at data structures, one tries do address quite a bit more than "contiguous memory", for example persistence, runtime updates and runtime lookups, ... see for examples: https://cstheory.stackexchange.com/a/1550

> It's hard for me to imagine any sane architecture where using more memory and having if fragmented rather than contiguous is going to perform as well as using a contiguous block of memory.

Yet many of the language runtimes are actually following the Lisp model. Just that it's there not the cons cell with two slots, but arrays and objects with slots. A Java program is a multitude of dynamically allocated objects with one or more slots, where many of the slots contain pointers to other objects. The memory management is done by the runtime (and, usually, its garbage collector).

> Sometimes they aren't. That doesn't fit the narrative that Lisp is god's gift to McCarthy who then gifted it to humanity, but it is reality.

Other than what you might think (and set up as a strawman), that's well known and the evolution of Lisp is also showing how the implementations had to deal with this problem.

The LISP 1 implementation introduced mark&sweep garbage collection (-> McCarthy). As a reaction to it Collins proposed "reference counting" as another way to manage&reclaim allocated memory.

Over time early Lisp programs (Macsyma is an example often mentioned) were problematic on multi-user time-shared machines. One reaction to that was to develop single user workstations for Lisp, where all the memory was available to a single user. There one then used large virtual memory, which still made memory access (for example during garbage collection) time consuming. A GC run of a large heap in VM could run for 30 minutes making the computer mostly unusable during that time. Vectors/Arrays and OOP objects then also used the model of dynamic allocation with many referenced objects (unless they were of some primitive types). Generational GCs, compacting GCs, hardware-supported GCs, etc. were invented and implemented. Full GCs through virtual memory heaps were rare then. The availability of cheaper RAM made it possible that more of the large heaps fit into RAM.

Lisp implementations early on added vectors and other data structures. But they still often need pointers. A vector of records (called structures in Common Lisp) is usually a vector with pointers to records. For example Common Lisp implementations usually will not provide vectors of inline records. A vector of fixnums OTOH will be a vector without pointers -> the fixnums will be stored directly in the vector.

In the end a Lisp heap is a huge graph of objects referencing other objects. This does not only effect CONS cells, but also vectors, arrays, records, ... -> they also reference other, non-primitive, objects via pointers.

That's not very different from any other language runtime which uses dynamic memory allocation and a heap of linked objects (-> Java, JavaScript, ...). The problem of dealing/avoiding non-locality and pointer-chasing is there, too.

> since he became inactive, though

Again you are digging yourself deeper into this hole. Now I'm really out.

> My objective here is not to attack you for your partisan support of Naggum

Look, I'm not supporting Naggum.

From the actual Lisp users I've met in real life, few were reading comp.lang.lisp, almost none was posting there. Very few would even know who the people there were.

The actual people in the Lisp community I admire are many - I could list 100 - they had a much greater impact on what was happening. Most of them never used comp.lang.lisp . Among women the participation rate was near zero!

Most of the interesting communication was in specific communities over mailing lists. I fondly remember SLUG (the Symbolics Lisp Users Group), the MCL mailing list, and several others.

comp.lang.lisp had the most negative impact by trolls, like the one Naggum was answering, which caused endless bullshit discussions and which had very destructive behavior. We never found a way to deal with that. What it killed in the end was that the time for Usenet was over, Google groups, spam, it was mostly irrelevant and a much more toxic Internet took over. See 4chan, various Reddit groups (though the Lisp subreddits at Reddit do better) and X for much worse communication systems. Plus Lisp was in a general demise in the 90s. Not the Usenet caused the demise of Lisp, but the demise of Lisp was causing a general lack of interest. Cause and effect were different of what you claim.

For you the contact to Lisp could be mostly net related (?). When I was learning at the university roughly up to 20 professors were in some way or another involved in Lisp projects. There were people I've seen in real life, often daily. There were many people with deep Lisp knowledge (and this was not even the US). You can bet that none of them used or cared about comp.lang.lisp. Same for the people in the projects and the students.

What you think was "the Lisp community", was just a popular online community of mostly random people, with less impact on what actually happened in the Lisp communities, then you believe.

"self-evidently correct" is not a good word if you want to discuss things with people who were actually active in the wider Lisp communities, including comp.lang.lisp, other newsgroups, various mailing lists, Lisp conferences, etc.

You are, sadly, digging yourself deeper into a hole. I'm not interested to follow you there.

Lisp programs often needed large amounts of virtual memory. Factor ten (VM size vs. RAM size) was not unheard. RAM was basically a cache for that.

German has this great word for your comment: "unterkomplex". The English "simplistic" might come near. comp.lang.lisp was not "the Lisp community", Naggum did not single handedly destroy it, you ignore the other participants, the general Usenet dynamics and its overall fate...

Message passing means that objects communicate only via messages with some payload. There is a sender and a receiver. Possibly there are many receivers in a broadcast message. Messages will run synchronous or asynchronous. The receiver decides what to do with the message and how to interpret it. If it does not understand the message, it can forward the message via inheritance, delegation or similar. The message implementation belongs to the class and/or object. There is a syntax for sending messages. The mental model of a developer is based on communicating objects.

In Common Lisp the object system uses generic functions, which assemble several related methods. Generic Functions are defined outside (!) of classes and objects. Generic Functions are both CLOS objects and functions. Generic functions are called with one or more arguments. The Generic Function decides which methods are run and in what combination. There is no equivalent to a receiver: CLOS generic functions dispatch over its arguments, all primary arguments. There is no equivalent of a privileged receiver object or a self. There is no special syntax for calling a generic function and they are themselves first class objects, which can be stored and passed around. The mental model of a developer is based on function calling of runtime assembled functions.

I would think that Scala is slightly more object-oriented than Common Lisp, given that one can mostly ignore OOP in some Common Lisp applications. Thus that part already (macros !) makes CL very flexible in expressing different ways of programming. CLOS without MOP has a range of possible use. I haven't thought about how consistent typical code actually is, using CLOS without MOP. My intuition would say that there is some common CLOS style, but maybe that's wrong. It would be an interesting question to look at, how CLOS code bases make use of its features and if they share common architecture principles. Actually there is not that many literature on software engineering principles for CLOS. For example see: "CLOS in Context: The Shape of the Design Space" by Bobrow&Gabriel&White.

https://www.dreamsongs.com/Files/clos-book.pdf

The MOP itself was thought to make CLOS programmable and thus to support a wide range of different object-oriented styles in a single language framework. In practice there is use of that, but it's not that common and actually also a bit complicated/tricky/challenging to program the object system in itself for different behavior.

Yes, there are a bunch of frameworks, which make use of inheritance and/or delegation.

Flavors (an early object-oriented system for Lisp, early 80s) provided multiple-inheritance, mixins, method combinations, etc. and a bunch of software made use of its features. Object Lisp was another OO system for Lisp, which one used for example in an interface builder, which made extensive use of delegation. The Common Lisp Object System later was designed improving on Flavors and LOOPS (from Xerox), switching away from message passing, adding multiple dispatch. Thus dispatch based on inheritance works over possibly multiple arguments.

Inheritance and delegation have a long history in Lisp going back to the 70s.

It's just that "it" has no defined standard. See https://github.com/pcostanza/closer-mop/blob/master/features... for the MOP differences on platforms which have support.

Closer to MOP is a library which tries to fix many/most of these differences.

https://github.com/pcostanza/closer-mop

> might not be able to achieve performance similar to a "zero-cost abstraction" language like Rust

That might be the case, even though SBCL might have some performance improving features, like a runtime compiler (which can be used to improve performance at runtime) and the capability to implement assembler extensions to the compiler from Lisp.

> There's an argument to be made for this approach, at least for software that's run on customers' computer. For server side software, I can see why having always-available debuggability is useful.

I see no reason, why this should not be useful on the client side, too. Many software systems on the client side have ways to extend them. Doing this in the main implementation language can have advantages, instead of doing it in a limited macro or scripting language.

> Most of the features I've described above work with C and C++ IDEs

In Common Lisp you won't need an IDE and you won't need to instrument the application with a debugger, because much of the features are built into the language runtime.

For example Common Lisp has an error handling system with resumeable exceptions.

Another example is the object system where the objects are updated on changes. For example I can load a new class description and the existing objects are changed to the new description: new slots, fewer slots, new superclass, removed superclass, ... This makes hot loading much more useful. Additional many calls to global functions are late bound, which again makes hot loading more useful.

A typical runtime also includes a code loader, a compiler and/or an interpreter, which makes updating code easy. All without an IDE or a foreign debugger.

This also means that when for example the resident compiler is called, the compiler is a part of the runtime. An error under compilation, will show me the compiler in the stack trace and the compiler itself can be live edited/updated, while I'm in a compilation error.

The effect is then, that by default programs are running inside a runtime, which always provides the features of interpretation, compilation, code loading, repls, debug repls, runtime type checks, OOP updates, ... One would need to do something to turn those things off.

In a typical C/C++ program, the compiler, debugger, IDE, linker, ... are not a part of the program's runtime.

> I wouldn't say reuse of functionality is a core aspect of OO

You'll find it in practically every object-oriented language, starting with Simula. Later Smalltalk, Actors, CLOS, C++, Self, Java, Objective-C, Python, JavaScript, ...

> making things "reusable" only to never be reused.

It's similar mind-numbing how much has been reused.

> not a core aspect of OO

One of the core aspects of OO is reuse of functionality, typically by inheritance or delegation. We can find one or both in most OO languages.

Part of the Common Lisp experience is tool building at the language level.

You could be digging a hole with your hands. But thinking a moment, somebody came up with the idea of a shovel. That made a huge difference.

Similar, tool building & usage at the language level can also make a huge difference. In larger applications this makes code much more compact and makes a person more productive.

One idea of using Lisp is to learn new thinking styles for programming. Thinking styles are not fixed. For example an early hurdle in programming / computer science is recursion. Lisp gave the teachers a tool to teach thinking of software development in terms of programming with recursive functions&datastructures, starting with linked lists and operations for it...

Unfortunately students often got the impression that Lisp is only good for learning new ways of thinking (in combination with a theory of recursive functions) and not for actual software development.

Common Lisp also provides type information, dynamic type information.

    CL-USER 72 > (defclass person () (name age))
    #<STANDARD-CLASS PERSON 80101598FB>

I can now ask Lisp to find the class object named person.

    CL-USER 73 > (find-class 'person)
    #<STANDARD-CLASS PERSON 80101598FB>

Now I've got a real class object, not just a pointer to dead code. I can inspect it, change it...

I'm adding a slot to an existing class:

    CL-USER 74 > (defclass person () (name age height))
    #<STANDARD-CLASS PERSON 80101598FB>

I can then ask for the slots of that class.

    CL-USER 75 > (class-direct-slots (find-class 'person))
    (#<STANDARD-DIRECT-SLOT-DEFINITION NAME 8220278CA3>
     #<STANDARD-DIRECT-SLOT-DEFINITION AGE 8220278CBB>
     #<STANDARD-DIRECT-SLOT-DEFINITION HEIGHT 8220278CD3>)

I can define a method for it.

    CL-USER 76 > (defmethod greet ((a-person person)) (format t "Hello ~a" (slot-value a-person 'name)))
    #<STANDARD-METHOD GREET NIL (PERSON) 801001B69B>

I now can retrieve the method object from its generic function.

    CL-USER 77 > (find-method #'greet () (list (find-class 'person)))
    #<STANDARD-METHOD GREET NIL (PERSON) 801001B69B>

That's live code exploration using dynamic types.

> You can do this with most major languages today as well.

Hmm, I'm in this graphics program on my Mac, how do I "break" the program and then interact with its Objective-C objects?

> I don't know how, or what made it more than what Slime might offer beyond nice access to the Mac toolbox).

GNU Emacs plus SLIME is a remote development environment for an external Common Lisp, using a text-editor-based environment.

Macintosh Common Lisp was an integrated GUI-based Common Lisp develop environment written in itself.

Those are two very different things.

> Flavors (Genera's object system)

CLOS and Flavors are both Genera's object systems. CLOS was actually co-developed by Symbolics. The core designers were Keene & Moon from Symbolics Inc., Kiczales & Bobrow from Xerox PARC. and DeMichiel & Gabriel from Lucid Inc.

The Meta-Object Protocol is optional and not a part of the standard.

The Common Lisp Object System does not use message passing.

One needs to use them in their personal lighting situations, not in a store.

I have both the current and prior iPad Pro version. IPS+Mini LED and the Tandem OLED. The IPS version was already great. The Tandem OLED is generally better, but often not by that much. It's also a bit brighter. Both are quite a bit better than the normal iPad displays.

I've seen the nano texture version of the Tandem OLED. Reflections are much less, but in brighter surroundings, the image is not that dark. But again, it all depends on the personal lighting situations.

> Given the development model of Lisp is modifying a live image of a running program

I would say it is modifying the running program itself. An image is the saved version of the heap. Saving/loading images is optional.

But with Lisp one can also compile source files to compiled files. In rare occasions it is also possible to compile whole programs.

it's similar, but Lisp was rarely "batch first" or "batch only", it was already coming from 100% interactivity and then adding compilation as subsystem. So running code during compilation is the default, for example with Lisp macros, which are normal functions. The software being compiled can add arbitrary code to the compilation environment.

The traditional idea "compiled language" usually means a language designed for mostly batch compilation -> the compiler is not a part of the (potential) execution runtime. "compile time" and "run time" are not the same. In Lisp it is allowed to be the same.