Comment by dan-robertson | Hacker News Reader

Comment by dan-robertson | Hacker News Reader

I’m not aware of any “full” uses of SICL. It’s compiler framework, cleavir is used by clasp (a somewhat new c++/llvm/boehm/ecl based implementation).

Long ago, I tried to build a CL->JavaScript compiler based on cleavir. It started ok and I had the definitions of some annoying things (eg lambda list parsing code gen) but I was caught by the lack of any interpreter [1] with any kind of acceptable performance, and I eventually gave up. I think there weren’t any JS based implementations at the time but that might not be the case anymore (I haven’t followed CL closely in a while). My ultimate goal was to have the ability to write web applications where the server and client could share code in the best way possible, including things like throwing closures from server code to be executed on the clients[2]. However that would in fact require also having a compiler to native code [3] and the design of cleavir was well suited to handling the sort of “dual compilation” that was required.

[1] you need an interpreter to evaluate macros. You cannot portably use the host lisp’s macro expansion system because you cannot use e.g. symbol types from the host lisp (due to issues with packages, interning and namespaces) and you cannot sufficiently control the global or compiler environment. The interpreter must be used for evaluating the bootstrapping code to execute macros in the target environment to create that environment. SICL did have a simple interpreter at the time (removed around then too) but it used much more memory than I could give it.

[2] I also wanted to support compiling some weird code which is not required by the standard like:

  '#.(let ((a 0))
       `(,(lambda () (incf a))
         ,(lambda () (decf a))))

(Here the source code itself contains a list of two objects which are functions closing over a common environment.)

[3] the above is one example of why a second native compiler was needed. Making bootstrapping/compilation adequately fast was another. In general marshalling objects is hard to do portably hence why I wanted a native target. Another complication is that for consistent semantics you need to only use (IEEE) double-precision floats in code that might be compiled to JavaScript but there isn’t even a portable way to get such a float!

You do not need an interpreter to evaluate macros. You can compile the code of the macro, same as you would any regular function, and call into it that way.

I think you’re missing something: if I compile the macro to JavaScript, I don’t have a way to run it because I am cross compiling. I can’t simply use the host lisps compiler because of reasons mentioned above.

Macro expansion is done in the host environment, not the target environment. You can compile the macro definition to a function taking a form and an environment. This would be something you arrange for within your compiler.

CLTL2 specifies a function `parse-macro ` which is available in most implementations. Portably you can call it using the `trivial-cltl2` package:

https://github.com/Zulu-Inuoe/trivial-cltl2

Robert Strandh has written both papers and code for handling environments:

http://metamodular.com/SICL/environments.pdf http://metamodular.com/clostrum.pdf

Either you are describing something that doesn’t work for cross-compilation or you are missing the point. Common Lisp is a hard language to compile because of macros. Consider the following code you might see in bootstrapping:

  (defmacro defun ...)
  (defun ...)

To macroexpand the second line you must have an environment with the defun macro definition from the previous line, which means that the code to modify the environment with that definition must have been evaluated too. This must be evaluated in the (emulated) target environment because (1) the host environment would otherwise clash with it, and (2) you need to write down the target environment as a build artefact. Evaluating these environment-modifying forms requires an interpreter you can run on the host. If you want a good programming experience for your standard library then this needs to be a reasonably capable interpreter.

But you also need an interpreter to evaluate code during macroexpansion. Three reasons:

1. You must emulate the target environment for correctness, eg JavaScript only has one type of float.

2. You cannot sufficiently interact with the host environment to use its own macroexpander (if you want to be portable). This is because compiling a macrolet requires extending the lexical environment with the macro definition so that calls (from eg the expander for setf) to macroexpand can use the definition. The only portable thing on the host that can evaluate a macrolet is the compiler/interpreter

3. You probably can’t use native types for things like symbols so the built in macro expansion is not sufficient.

That the runtime environment and the compilation environment, insofar as they are distinct, are not able to communicate seems like a deficiency in the implementation. Lisp is meant to be interactive.

Do you not expect to be able to implement 'eval'?

To a large extent, they can communicate: the compile time environment when compiling one form depends on the runtime environment modifications made by evaluating the previous form. This is what makes cross compiling lisp hard.

Reply on news.ycombinator.com