lispm

> The second approach is Lispy, but that shifts the responsibility of creating abstractions to users.

"You" can provide those syntax abstractions as macros. "You" can also use them yourself. But they then can be embedded in Lisp, you don't need to provide a new programming language version or even a new language & implementation with those abstractions. The user may be sophisticated enough to provide his/her macro abstractions.

> None of the assignments my classmate struggled with required him write code that processes code

That's not what I meant. What Lisp with s-expressions already makes more difficult is that the data syntax is used to express programs.

    (defun foo (list1 list2)
      (quote (defun foo (list1 list2) (list list1 list2))))

-> A symbol can be an identifier and just a piece of data.

Above, the first DEFUN is a standard identifier of the programming language Common Lisp. The second DEFUN is just a symbol.

The first DEFUN form is a program. The second DEFUN expression is just data. Thus it makes no sense to indent the second DEFUN expression like a program. We don't know if it is a program, it could be just anything.

-> A list can be data and a program.

This specific double nature of built-in data structures, here symbols and lists, is already a hurdle to get over.

The developer has to know which lists are data lists, which are forms and which are expressions in inside a form.

    (cond ((quote (a))))

->

    (cond ((quote (a)))) is a form, which is meant to be evaluated

    (a) is data

    ((quote (a)))  is a clause of COND, which is neither data, nor a form to be evaluated

Thus the user has to understand that in Lisp these lists serve different purposes. Which is which depends on the context and the Lisp syntax & evaluation rules.

This then even may make functions possible which contain their own definition as data:

    CL-USER 9 > #1=(defun foo (list1 list2)
                     (quote #1#))
    FOO

    CL-USER 10 > (foo '(1) '(2))
    #1=(DEFUN FOO (LIST1 LIST2) (QUOTE #1#))

Lisp early on exposes its dual nature of code and data. Even in interactive environments. This ever-present meta-level is unusual.

> Start to develop macros where you do not know how arguments are evaluated or when, half of which you do not know and good luck figuring out how to use and combine things with functions/macros in other places.

I would guess a team would be so clever to develop maintainable macros.

In many places it's actually the very idea not to know how the thing is evaluated. One writes more descriptive software and does not specify execution details.

You can write unmaintainable software in many ways. Macros are a tool for code generation. How to write and use them can be learned, just like people learn to maintain large class libraries in Java.

Really, there are many of examples of Lisp code bases which make heavy use of macros and where the code was written by tens to hundreds of people.

For example much of the user facing Lisp functionality itself is macros.

Something like CLOS has much of its functionality exposed to the programmer as macros, which sit on top of functions and OOP. It's a domain specific extension to Common Lisp for OOP.

GUI libraries use macros, tools for 3d modeling use macros, ...

Whenever groups of people use code, macros are a tool to provide common language extensions. Like any tool, it needs to be learned how to use it.

ECL and ABCL are Common Lisp implementations. They are both interactive and both don't use images.

Unless one saves and load images, one is not using image-based development. All one is using is the interactive nature of Lisp, via a dialog interface, here a REPL. Lisp uses late binding, which makes it possible to update function definitions in interpreted and compiled software.

The base for interactive Lisp development is:

* a procedure to read data

* a procedure to evaluate data as code: EVAL

* printing

* a Read / Eval / Print loop

* additionally Lisp provides an incremental compiler and a file compiler

* LOAD to load code

Still a typical Common Lisp application or library consists of files. ASDF is the moral equivalent of "make" and runs the file compiler. Quicklisp will transfer Lisp software in the form of files to your local machine, compile the files there and load them.

So far we have not used "images". If you check the ANSI Common Lisp standard, there is not a single word about "images" in the standard definition.

GNU Emacs wasn't written for Lisp. It was developed as an extensible text editor. Its extension language was also the implementation language for a large part of the software. Much of the application domain of GNU Emacs has nothing to do with Lisp. People edit their files with it or integrate with tools.

> Also, the superior interactivity means you have an image. But when I create software I organize things in files, I run them from scratch. In Lisp everything becomes (or tends to become) kind of a stateful environment. It takes more reasoning to reason about current state.

Interactivity has nothing to do with an "image". An image is a dump of the memory heap. That's a feature found in several, but not all, Lisp implementations. There are Lisp implementation which support image-based development (like Interlisp). But I doubt that you have used that.

What many Lisp systems actually often are: they are interactive with a resident language environment. There are many examples for non-Lisp implementations of that idea: APL, BASIC, Prolog, Smalltalk, Python, Ruby, R, ...

Still, software in most Lisp projects is organized in files. Something like Common Lisp includes a compiler, which compiles files and a loader which loads files (compiled and source). But the Common Lisp runtime usually includes a compiler and an interpreter and one then can develop the application interactively while it is running. The code is still written in files.

> but the macro/DSL parts are what also prevents it from being easily picked-up by teams.

That's a myth. Actually macros are BEST used in teams.

Small teams of 5 to 100 people can easily define/use language extensions, document them and reuse them in their application software. Larger Lisp software benefits most from macro definitions, since they enable development of portable language extensions incl. embedded domain specific languages, without the need for low-level implementation hacking. -> the result is very compact user level code.

> "Hacker News later got rewritten entirely in another language"

AFAIK Hackernews is still written in Arc.

> "Okay, if Lisp is so powerful as you say it is, then give me one example where Lisp is used? More importantly, give me one example where Lisp was used and didn't get rewritten in another language later?", I hear no concrete example given by the Lispers.

Maybe the problem are statements like that, which sound a bit like trolling, which people are tired of? There is more than one example of software written in Lisp and still maintained in Lisp.

> But this means that `cond` is not a function anymore because it could be that > A popular notion is that Lisp has no syntax. People also say Lisp's syntax is just the one rule: everything is a list expression whose first element is the function/operator and the rest are its args.

That's wrong. Lisp has functions, special operators (like IF) and macros. Also any data-object is also valid Lisp.

Valid Lisp examples:

    3          -> number
    "string"   ->
    hello      -> variable
    (print "hello world")  -> function call

special forms

    (if (> a 10) 1 0)      -> no more than three args,
                              arg 2 & 3 are evaluated depending on first argument value

    (let ((var1 20)        -> definition of local variables
          (var2 22))
      (+ var1 var2))

    (quote hello)          -> definition of a literal object

macros

    (defun foo (arg1 arg2)       -> defun is macro, which defines global functions
      (if (> arg1 arg2)
          1
          0))

> But this means that `cond` is not a function anymore because it could be that for two different inputs, it returns the same output.

A function can return the same output for different inputs.

    (defun foo (args)
      0)

Above returns 0 for all arguments. It's a valid function.

> So essentially, my understanding so far is that Lisp has list expressions, but what goes on inside those expressions is not necessarily following one unifying syntax rule—it actually follows many rules. And things get more complicated when we throw macros in the mix: Now we have the ability to have literally any syntax within the confines of the list expressions, infinite syntax!

That's true.

My basic mental model of SETF was always relatively simple. We were told that in university, many decades ago.

Lisp has traditionally a bunch of different datastructures. Like cons cells with CAR and CDR accessors. RPLACA and RPLACD were the setters, from a time when characters were expensive, so that identifiers were short and abbreviated (RPLACA would be short for Replace CAR). Then there were property lists with accessors GET (-> Maclisp) or GETPROP (-> Interlisp). The setter was PUTPROP.

So for any datastructure we need to learn both a getter and the corresponding setter.

With SETF in Common Lisp we only need to know the getter. SETF knows for each getter a corresponding setter.

So for a datastructure FOO we don't need to know both GETBAR and BARSET!.

The getter is (GETBAR SOMEFOO) and the setter is (SETF (GETBAR SOMEFOO) SOMETHINGNEW). SETF then is a macro which knows how to set the thing.

Advantage: we don't have to learn ancient functions like RPLACA with calls like (RPLACA CELL 42), but we can write (SETF (CAR CELL) 42). Take the accessor and wrap a SETF around it, include the form for the new value.

In reality there is a bit more magic involved.

The first Lisp implementation in 1960 already had function tracing.

Technically this works in Scheme, but not in Lisp.

In Common Lisp one would need to write

    (funcall (if (< 1 0) (function +) (function *)) 42 100)

Multics Emacs and the Lisp Machine EINE/ZWEI/ZMACS were also implemented in Lisp, they did not use an embedded scripting Lisp implementation. There is no boundary between an implementation and scriptling language, just one Lisp (Maclisp for Multics Emacs and Lisp Machine Lisp (aka ZetaLisp) for EINE/ZWEI/ZMACS), which in the case of the Lisp Machine also was used to implement the rest of the operating system and all its applications.

It could be a helper-function which gets called by a macro expansion at compile-time. One would define the feature as a function which is available at compile-time and the macro calls it during expansion.

CLASP in C and C++ via LLVM.

LispWorks and Allegro CL as shared libraries in various environments. LispWorks in iOS and Android Apps.

SBCL, I think, now possibly works also as a shared library.

That's great that VB and Delphi had one in 97, but the Interface Builder for NeXT was released in 1988.

If you haven't seen it, the Interface Builder got its inspiration from this Interface Builder written in Lisp: https://vimeo.com/62618532 The version in the video actually ran on a TI MicroExplorer Lisp Machine NuBus board in a Mac. There were other ports which ran directly on the Mac in Lisp.

The thing was shown to Steve Jobs and he hired its main developer, Jean-Marie Hulot.

> There are multiple token types and they always combine / apply in the same ways, there are no special forms like "how to define a function" or "how to write and if else". In Rye calling if or eihter (if/else) is no different than calling any other function.

Not in Lisp. In Lisp IF, COND, ... are either special operators or macros which expand to more primitive special operators. They are not functions.

The syntax for COND is:

    cond {clause}* => result*
    clause::= (test-form form*)

For example:

    (cond ((> x 1) (print 'larger) 1)
          ((= x 1) (print 'equal) 1)
          (t       (print 'smaller)))

COND is not a function. It is a macro. In a function call all arguments would be evaluated. But COND does not work that way.

COND has a list of clauses. During evaluation, the first clause is looked at. It's first subform then is evaluated, here (> x 1). If that is true, then the following subforms are evaluated left to right and the values of the last form is returned. If the first subform form was not evaluated to true, the evaluation goes to the next form and repeats the process on the subform.

* COND is not a function

* COND has a special syntax

* COND has a special evaluation rule

The same is the case for defining functions. In Lisp the operator to define a global function is called DEFUN. DEFUN is a macro, not a function. It has special syntax (for example the second element in a DEFUN form needs to be an unevaluated function name, which usually (I omit some detail) needs to be a symbol (which will not be evaluated)...

Again, DEFUN is not a function, it has special syntax and special semantics with side effects during compilation.

> SBCL is mediocre at best at optimizations but most people do say that.

I would think that it takes a bit more knowledge to judge that.

You'll need to understand a bit more how to declare types to achieve better results.

> Are you serious? This should be 1, max 2 instructions, with no branches and no memory use.

Sure. ---> One could add two fixnums and get a bignum.

    CL-USER 31 > (fixnump (+ MOST-POSITIVE-FIXNUM MOST-POSITIVE-FIXNUM))
    NIL

As you see here, adding two fixnums can have a result which is not a fixnum.

Yes, Common Lisp does by default determine whether to return a bignum or fixnum.

The machine code you've shown takes care of that.

Let's see what the SBCL file compiler says:

    CL-USER> (compile-file "/tmp/test.lisp")
    ; compiling file "/tmp/test.lisp" (written 11 JUL 2024 10:02:04 PM):

    ; file: /tmp/test.lisp
    ; in: DEFUN FX-ADD
    ;     (DEFUN FX-ADD (X Y)
    ;       (DECLARE (OPTIMIZE (SPEED 3) (SAFETY 0) (DEBUG 0))
    ;                (TYPE FIXNUM X Y))
    ;       (+ X Y))
    ; --> SB-IMPL::%DEFUN SB-IMPL::%DEFUN SB-INT:NAMED-LAMBDA 
    ; ==>
    ;   #'(SB-INT:NAMED-LAMBDA FX-ADD
    ;         (X Y)
    ;       (DECLARE (SB-C::TOP-LEVEL-FORM))
    ;       (DECLARE (OPTIMIZE (SPEED 3) (SAFETY 0) (DEBUG 0))
    ;                (TYPE FIXNUM X Y))
    ;       (BLOCK FX-ADD (+ X Y)))
    ; 
    ; note: doing signed word to integer coercion (cost 20) to "<return value>"
    ; 
    ; compilation unit finished
    ;   printed 1 note


    ; wrote /tmp/test.fasl
    ; compilation finished in 0:00:00.031
    #P"/private/tmp/test.fasl"
    NIL
    NIL

Well, the SBCL compiler does tell us that it can't optimize that. Isn't that nice?!

If you want a fixnum to fixnum addition, then you need to tell the compiler that the result should be a fixnum.

   (the fixnum (+ x y))

Adding above and now the compiler no longer gives that efficiency hint:

    CL-USER> (compile-file "/tmp/test.lisp")
    ; compiling file "/tmp/test.lisp" (written 11 JUL 2024 10:03:11 PM):

    ; wrote /tmp/test.fasl
    ; compilation finished in 0:00:00.037
    #P"/private/tmp/test.fasl"
    NIL
    NIL

> Trying not to beat a dead horse, but I recognize there may be many reasons people come up with for writing a new library instead of reusing one. My point is that it has a business impact to do it that way, because unless that library is open-sourced, the next time it happens it will be an overall loss for the industry due to solving the same problem again and again in slightly different ways.

One of the reason to use Lisp was to leapfrog the competition. See for example: https://paulgraham.com/avg.html Applications were written in novel areas by small teams. The commercial Lisp vendors had their development environments bought and be used by these teams, but they could not post success stories, because they were not allowed to mention the customer and its application. Here often substantial and complex software had been written, at the core of some new business venture. The reason this software code could not be used by, then was not that it easy to copy, but that the developers did not want to give their competitive advantage away.

For Lisp libraries -> if any library domain gets complex, there will be only a few ones which actually get used, because it was non-trivial to write. People might try to write simpler or new ones, but the better ones are hard to replicate and Lisp developers will tend to use those. For example the Lisp cookbook recommends one Regular Expression library: https://lispcookbook.github.io/cl-cookbook/regexp.html That's what most developer tend to use - it is widely ported, written in good style, well tested, ...

A strange thing in the Common Lisp world is that new developers stumble of old libraries. They expect libraries with lots of bug fixes and a long maintenance log. But the library hasn't much touched in since some years.

They detect then, that this library is "done". It has reached a state where not much needs to be done. This is also an effect of Common Lisp, where the language is mostly stable for several decades and code can be used for a long time without the need for a new attempt.

> In the 1980s I felt like I was all alone sometimes developing on my Lisp Machine - I often would write utilities that I wished I didn’t have to write myself.

The early .com companies on the Internet were all deeply involved in Lisp/AI companies: Symbolics, BBN, Thinking Machines, MCC, Xerox, SRI, ...). Plus a bunch of universities (MIT had hundreds of networked machines running some Lisp, including Lisp Machines).

Many of the R&D groups were on the Internet. Larger amounts of code were for example shared via tape. For example the Symbolics User Group already shared code. The CMU CL wrote their Lisp implementation as public domain and shared all their implementation code: Lisp, editor, UI, ...

Lots of old code from that time is all collected in the CMU AI Archive, which was established early 90s: https://www.cs.cmu.edu/Groups/AI/0.html

At that time a lot of new stuff was developed, like CLX, the X11 interface for Common Lisp. Every Lisper on X11 was using it. TI developed CLUE and CLIO on top of CLX. LispWorks developed their gui toolkit on top of CLUE and CLX.

Sure, there were lone developers but Lisp in the 80s was widely used in universities and R&D. All were networked. MCC had 200 Lisp Machines in their network, IIRC. MIT more. SRI similar. Xerox. BBN even developed their own Lisp OS + machine. Symbolics had support for DECNET, TCP/IP, IBM Mainframes, ...

The Common Lisp standard, developed in the 1980s was probably the first developed with heavy e-mail support.

There was a lot of stuff shared.

> I have never heard anyone contest the 'Lisp Curse' claim

You could have asked in the community for opinions.

> This is because Lisp makes it very easy to write your own libraries from scratch;

That's not generally true. Some libraries are maintained for decades, since it would be difficult to reproduce them.

The reasons for writing new libraries, instead of an existing one are more like:

* disagreement with coding styles

* very different underlying platform

* need for a different license

* lack of maintainers for code written decades ago

> people who use Lisp tend to do that instead of reusing third-party dependencies.

I don't think that's the case. There are widely used libraries for a bunch of things and having a stable and standardized language makes it actually easier to write platform independent code and enables code sharing.

There are reasons why people write their own libraries, often because existing libraries were written for different platforms (especially the operating system).

In the free software area of Common Lisp, a system like SBCL is popular because of the network effect in a small niche of Lisp programs. Libraries tend to be better supported in SBCL -> more use for SBCL.

> people who use Lisp tend to do that instead of reusing third-party dependencies

Is there any actual evidence for this?

> Lisp purists will argue M-expressions are unnecessary and S-expressions are all you need.

I would argue that S-Expressions turned out to be extremely practical for developing Lisp software. A layer with different syntax makes it more complex to use.

> Maybe if the M-expression idea had been pursued more seriously, Lisp might be more popular today.

Maybe, maybe not. People have discussed it endlessly, but there is no conclusion.

Currently some people in the Racket community try to make Racket more "popular" (widely used, ...) by providing a new syntax.

> And how do you think the cost in the bill is determined?

A cost of a bill is determined by the AMOUNT a buyer buys from a certain seller at a certain PRICE, plus the extra costs (like taxes, fees, ...).

...

FEXPRs existed around 1960. Macros slightly later.

MIT AI Memo 57, Timothy P. Hart, MACRO Definitions for LISP, October 1963

http://bitsavers.informatik.uni-stuttgart.de/pdf/mit/ai/aim/...

> However, the more data you feed into an LLM, the smarter it should be in the response.

Is it that way? For example if it lacks a certain reasoning capability, then more data may not change that. So far LLMs lack useful ideas of truth, it will easily generate untrue statements. We see lots of hacks how to control that, with unconvincing results.

You might want first try to understand what the author means by "Metasystem". It's not about "meta programming", but a "meta system". The LISP I implementation is a "system", it was "a programming system for the IBM 704 for computing with symbolic expressions.". This is literally the the first sentence of the LISP I manual.

"fully developed" does not mean "true", more like "complete".

For him a metasystem was a language system (just like a Lisp System is a List Processor System. LISP was actually the abbreviation for List Processor, not for List Processing), like a computer algebra system (CAS) with its own language, built on top of an implementation language. The first SCHEME system on top of a MACLISP system would be a Metasystem. The language SCHEME was not an extended Maclisp. The first ML would also be a Metasystem on top of, again, Lisp. ML was a complete Metasystem (funny, ML -> Meta Language), it had its own syntax, its own semantics, etc. It was first used as a part of a theorem prover, LCF.

Both SCHEME and ML were "fully developed", since they had their own syntax, had their own user interaction (-> a Scheme REPL or a ML interface). A CAS like REDUCE also has its own implementation language with its own syntax (the language is called RLISP), its own user interaction, etc.

Often these Metasystems would not be used from within Lisp (as an embedded language), but in a way, where the user does no longer interact with Lisp directly. For example in Macsysma, one gets into its own interaction interface and only can "escape" to Lisp via special commands or functions.

In Expert Systems implemented on top of Lisp in the 80s (and later), we have seen the same: some are embedded into Lisp (like KEE, the Knowledge Engineering Environment from Intellicorp, which can be loaded into a Lisp system, and developed as Lisp code) and some provide complete new programming environments (like GENSYM, an Expert System for real-time process control), with also a new language.

Herbert Stoyan then describes what kind of Metasystems had been developed in LISP until 1980.

His book was written, while he was still in the GDR (the German Democratic Republic, behind the "Wall"), in the late 1970s. It was published in the GDR. In 1981 he managed to go to West Germany, the Federal Republic of Germany. Since 1986 he was professor at the University of Konstanz, Germany. Later he was professor for AI in Erlangen/Germany. John McCarthy was visiting him there.

>> What are "unstable electricity bills" ?

> https://www.epexspot.com/en/market-data?market_area=DE-LU&au... Today Germany a peak of 2300€/MWh at 6-7, and a negative of -0.05€ at 12-13.

That's a price, not a bill. It's an auction. Electricity is traded there.

>> Changing market prices are the reality in many markets.

> For a market to be competitive, it must also be predictable. If energy costs vary drastically—€1000/MWh one day and €0 the next—it poses a severe challenge for companies. Such unpredictability discourages investment and may cause industries to relocate.

That's nonsense. "Markets" don't need to be predictable. There are lots of markets, where predictability plays a minor role. There are also renewable energy forecasts, just like there are weather forecasts.

In the future we will have much much more electricity capacity from renewable. Lots of business will then take advantage of that.

That's a completely new market.

> Imagine an energy-intensive company unable to operate profitably due to volatile energy prices.

energy intensive companies have long-term electricity contracts. Additionally many of these companies can steer their demand.

> Paid employees ready to work might have to wait because high energy costs could lead to financial losses.

What?

>However, this interconnectedness does not solve the problem. If Germany's electricity prices soar, they might rely on cheaper Swedish electricity, potentially saturating Sweden's supply and raising costs for Swedes. This scenario recently played out, prompting the Swedish government to halt plans for a new power cable to Germany.

That's all yesterday thinking. Germany is not only connected to Sweden (which it is indirectly) and countries/companies can regulate the export of electricity.

> The consequences are counterproductive. Germany's unstable energy market negatively impacts its citizens and neighboring countries. Rather than increased connectivity, the opposite approach might be more beneficial.

Sure not. It's the fear of modern technology.

> While the transition to renewable energy is essential, it is not without its challenges. Recognizing the limitations of renewables is crucial for anyone genuinely committed to this goal.

Sure there are challenges. Challenges are everywhere. We here in Europe, for example have the challenge to get rid of the energy dependence from Russia, which has served Europe (and Germany) with gas, oil and nuclear (fuel production, technology, fuel reprocessing, ...). Russia has used its energy industry to blackmail Europe. That's a challenge.

But you assume "limitations" of renewable energy which simply are not there or only temporary. For a lot of these "limitations" we will see solutions, like large interconnected grids, demand steering, usage of over-capacity in fuel production, large batteries, ...

We have seen this "limitation" thinking decades ago, when the German electricity industry (mostly coal and nuclear) claimed that renewable energy could only provide a few percent energy share. The grid would be unstable. The reality: the German grid is still super reliable and we approach 60% electricity from renewable energy - in a country with very little hydro capacity.

> Are you being paid to promote renewables? Many parts of your argument sound like poorly crafted commercials.

I had the same impression of your (in my opinion naive) nuclear talking points.

Hackernews (ran by a venture capital firm Y Combinator) is best for people who can extrapolate trends into the future. Energy policy is a field of long term industry trends. Changes can take decades. Currently we see that nuclear changes very slowly if at all, renewable still accelerates.

Don't bet the future on a stagnating energy technology.