Comment by lispm | Hacker News Reader

Comment by lispm | Hacker News Reader

Parent Full thread

> Also, the superior interactivity means you have an image. But when I create software I organize things in files, I run them from scratch. In Lisp everything becomes (or tends to become) kind of a stateful environment. It takes more reasoning to reason about current state.

Interactivity has nothing to do with an "image". An image is a dump of the memory heap. That's a feature found in several, but not all, Lisp implementations. There are Lisp implementation which support image-based development (like Interlisp). But I doubt that you have used that.

What many Lisp systems actually often are: they are interactive with a resident language environment. There are many examples for non-Lisp implementations of that idea: APL, BASIC, Prolog, Smalltalk, Python, Ruby, R, ...

Still, software in most Lisp projects is organized in files. Something like Common Lisp includes a compiler, which compiles files and a loader which loads files (compiled and source). But the Common Lisp runtime usually includes a compiler and an interpreter and one then can develop the application interactively while it is running. The code is still written in files.

> but the macro/DSL parts are what also prevents it from being easily picked-up by teams.

That's a myth. Actually macros are BEST used in teams.

Small teams of 5 to 100 people can easily define/use language extensions, document them and reuse them in their application software. Larger Lisp software benefits most from macro definitions, since they enable development of portable language extensions incl. embedded domain specific languages, without the need for low-level implementation hacking. -> the result is very compact user level code.

> Interactivity has nothing to do with an "image". An image is a dump of the memory heap. That's a feature found in several, but not all, Lisp implementations. There are Lisp implementation which support image-based development (like Interlisp). But I doubt that you have used that.

Actually I was unconsciously talking about the typical Common Lisp implementations I have used.

> Still, software in most Lisp projects is organized in files.

Yes, sure, but I am not sure the combination of image, promoting interactive development mixes well with do a git commit and typical CI pipelines. Interactivity encourages a flow where you "instantly see" results and experiment, whereas a compile-edit-runtests follows more naturally with more structure in test cases, etc.

Of course they have their uses, and their very useful use cases. But I do think they should not be abused and I also think they should be the last thing to use, when nothing else will work.

ECL and ABCL are Common Lisp implementations. They are both interactive and both don't use images.

Unless one saves and load images, one is not using image-based development. All one is using is the interactive nature of Lisp, via a dialog interface, here a REPL. Lisp uses late binding, which makes it possible to update function definitions in interpreted and compiled software.

The base for interactive Lisp development is:

* a procedure to read data

* a procedure to evaluate data as code: EVAL

* printing

* a Read / Eval / Print loop

* additionally Lisp provides an incremental compiler and a file compiler

* LOAD to load code

Still a typical Common Lisp application or library consists of files. ASDF is the moral equivalent of "make" and runs the file compiler. Quicklisp will transfer Lisp software in the form of files to your local machine, compile the files there and load them.

So far we have not used "images". If you check the ANSI Common Lisp standard, there is not a single word about "images" in the standard definition.

> That's a myth. Actually macros are BEST used in teams.

When there are some and well-known, yes. Start to develop macros where you do not know how arguments are evaluated or when, half of which you do not know and good luck figuring out how to use and combine things with functions/macros in other places.

They are totally double-edged and in many occasions more trouble than people would admit bc they fall in love with them and how terse they can look. However, scaling this without additional walls and freely is, in my experience, very difficult.

> Start to develop macros where you do not know how arguments are evaluated or when, half of which you do not know and good luck figuring out how to use and combine things with functions/macros in other places.

I would guess a team would be so clever to develop maintainable macros.

In many places it's actually the very idea not to know how the thing is evaluated. One writes more descriptive software and does not specify execution details.

You can write unmaintainable software in many ways. Macros are a tool for code generation. How to write and use them can be learned, just like people learn to maintain large class libraries in Java.

Really, there are many of examples of Lisp code bases which make heavy use of macros and where the code was written by tens to hundreds of people.

For example much of the user facing Lisp functionality itself is macros.

Something like CLOS has much of its functionality exposed to the programmer as macros, which sit on top of functions and OOP. It's a domain specific extension to Common Lisp for OOP.

GUI libraries use macros, tools for 3d modeling use macros, ...

Whenever groups of people use code, macros are a tool to provide common language extensions. Like any tool, it needs to be learned how to use it.

> You can write unmaintainable software in many ways. Macros are a tool for code generation. How to write and use them can be learned, just like people learn to maintain large class libraries in Java.

Macros are inherently more complex: they can transform ASTs, they can evaluate in several ways, they can inject variables in the same scope, they caan use the surrounding scope in surprising ways.

Functions have very well-defined input and output marks on it, which gives more structure.

I think you will agree that while macros are more powerful, it is way easier to write code more difficult to maintain and look at than with functions.

> Really, there are many of examples of Lisp code bases which make heavy use of macros and where the code was written by tens to hundreds of people.

I wonder when why people move to "inferior" languages like Python, Java or Go. Because they give you MORE structure, not construction material. The construction material can be used in so many clever ways that it is not even funny.

If there is a reason why people move to less powerful languages for team development that is because in some ways you find it easier.

In theory, Lisp is super powerful. It is true, I do not deny that. But it looks to me like macros are most of the time more trouble than not. With higher order functions and without macro abuse you can have more structure and code that maybe is not as terse or "good" at some point, but clearly readable and without any suprise in scope, hygiene or many things that can go deeply buried.

I would not remove macros from Lisp, though. But their use should be limited. I heard some reports (sorry I do not remember the source) that macros usually end up in DSLs that only the makers understand or that are at least difficult to understand for others. For sure there are ways to mitigate it with better docs and others, but they are still macros.

> Something like CLOS has much of its functionality exposed to the programmer as macros, which sit on top of functions and OOP. It's a domain specific extension to Common Lisp for OOP.

Yes: well-known, a few and well-documented. Now give that power to literally every programmer and let them use it freely. Then the cognitive overhead is unavoidable, like with anything else. But with macros things can get way more obtuse than with functions and a few pre-made mechanisms in a language that everyone understands.

Have you worked on a professional software engineering team where every member was able to do things "freely"? That's a project management failure, not a language failure.

Usually there are rules. But it is way easier to gollow Java or Go than Lisp full of macros or random memory-management C++ bc those are possible. In Java it is just more restricted , hence more streamlined.

Reply on news.ycombinator.com