Ask HN: Lisp eval vs. Lisp macros. Are they the same underlying concept? | Hacker News Reader

Ask HN: Lisp eval vs. Lisp macros. Are they the same underlying concept?

10 pointsbehnamoh11 comments

Is my understanding correct that Lisp's powerful macro system stems from the ability to write the eval function in Lisp itself? From what I gather, Lisp starts with a small set of primitives and special forms—seven in the original Lisp, including lambda. I recall Paul Graham demonstrating in one of his essays that you can build an eval function using just these primitives. Those primitives are typically implemented in a host language like C, but once you have an eval function in Lisp, you can extend it with new rules. The underlying C interpreter only sees the primitives, but as a programmer, you can introduce new syntax rules via eval. This seems like a way to understand macros, where you effectively add new language rules. I know Lisp macros are typically defined using specific keywords like defmacro, but is the core idea similar—extending the language by building on the eval function with new rules?

https://gigamonkeys.com/book/macros-standard-control-constru...

https://www.lispworks.com/documentation/HyperSpec/Body/f_eva...

They are subtly different. Macros are the only way to process s-expressions without first calling eval on them. This is necessary because without macros you could only generate and eval quoted code. Also practically it's cumbersome to work with everything being quoted when writing lisp code generators.

Without macros you could implement eval still but your internal lisp implementation could only work on quoted s-expressions, there would be no way to get back to the base unquoted level of lisp code (assuming you can't use the primitive eval, since you're trying to implement lisp in lisp).

Another use case is implementing a shortcutting boolean and function. You can't do the shortcutting without macros because all of the arguments get eval'd before passing to your and function.

eval and macros both deal with code-as-data (homoiconicity), but they serve very different purposes and work at different times in the program lifecycle:

eval

Runs at runtime.

Takes a data structure (usually a list) and evaluates it as Lisp code.

Example: (eval '(+ 1 2)) ; => 3

Use case: Dynamically construct and run code while the program is running.

Macros

Runs at compile time (or macro-expansion time).

Transform Lisp code before it's evaluated. They return new code (a Lisp form) to be compiled/evaluated later.

Example: (defmacro my-unless (condition body) `(if (not ,condition) ,body))

(my-unless (= 1 2) (print "Not equal")) ; expands to (if (not (= 1 2)) (print ...))

Use case: Extend the language by defining new syntactic constructs. Enables powerful DSLs and optimizations.

Given an eval primitive function that's a black-box, using it does seem somewhat similar to the use of a macro, in that a kind of "double evaluation" occurs (First, eval's argument is evaluated. Then the result of that is evaluated).

When a macro is used for any given purpose, what happens is a bit more general because it first "does something with" its argument (rather than necessarily straight-up evaluating it). Then the result of that is evaluated.

If you have to ask about macros, you REALLY dont need macros.

Macros are really only instructions for the compiler, how to compile things faster.

The syntax improvement aspect is minuscule, because Lisp has no actual syntax perse.

Hey Grok: Write commonlisp macro "test", which is usually an addition, but when two parameters are already numbers, it is the sum of those numbers.

  (defmacro test (a b)
    (if (and (numberp a) (numberp b))
        (+ a b)
        `(+ ,a ,b)))

These are very strange statements, coming from you. Please teach the kids properly! :)

Bad memories. Some malformed macro may evaluate differently than compile and the problem is impossible to find.

Hey Grok: Does commonlisp have some mechanism to prevent malformed macro to do things globally, so that eval works differently than compile?

  Grok: Yes.
  < 5 pages of semi-incomprehensible explanations omitted >

This is a feature. It is something you want sometimes and don't want at other times.

Macros can stage calculations to compile time. Compile time can happen in a completely different environment from run-time, such as on a different machine. It only happens once, whereas run-time can repeat many times.

A macro can be designed to that it opens a specific file, reads the content and generates some code based on that (or simply includes the file as a literal constant). That file exists only on the build machine, perhaps part of the source tree of the program. Thus, compiled code containing the macro call can run anyhere, but source code containing the macro cannot be evaled anywhere.

Different problems.

If you look at McCarthy's first paper [1] it becomes clear (after a long while) that eval does not have to support macros.

The ideas in Common Lisp on macros and evaluation (eval) are (conceptually "far away" from McCarthy's paper, perhaps more developed):

1. Read-time with "#." and #'set-dispatch-macro-character

2. Macroexpansion-time with #'macroexpand and #'defmacro and (macrolet …)-forms and (symbol-macrolet …)-forms

3. Compile-time (optional for evaluators) with #'compile and #'define-compiler-macro

4. Load-time with (load-time-value …)-forms

5. Run-time with #'funcall and #'eval (conceptually) and #'defun and (lambda …)-forms (conceptually)

The trick is that eval is called recursively and depth-first along the abstract syntax tree (AST). Function evaluation is depth-first left-to-right. Macros and special-forms are for not depth-forst and left-to-right. Loops need to evaluate (incf i) or "i++" possibly more than once, exactly once, or not at all. "If" only evaluates either its second or its third argument, but never both.

Note, how #'set-dispatch-macro-character, #'macroexpand, #'compile, #'load-time-value, and #'eval are all functions. They are "executed at (their own) run-time."

The difference bewteen compile-time and load-time becomes obvious in parenscript. In parenscript compile time has symbolic-expressions (s-expressions) and parenscript-forms a.k.a. "valid parenscript in parentheses" while load-time already has JavaScript code. In Lisp all of this stiff just happens to be Lisp objects. Strings at read-time (symbolic-expression), lists for macroexpansion (macros are like user-defined special forms) [macroexpansion turns them into valid common lisp a.k.a. Common Lisp forms], and internal Lisp objects like compiled functions for the compiler, "loader", and run-time.

Note, that the issue of having two namespaces, that of values and that of functions, is a separate issue. Here is where macros and functions look the same, because their names are at the beginning of a "form", e.g. (with-open-file …). With-open-file never directly evaluates its first argument: the list of arguments for function open.

A "form" is a list that is meant to be evaluated. If it is valid Common Lisp code, then it is a lisp form. If it user-defined, then it better be a function call or a macro call. And the macro call has to translate it into either the call of a user-defined function or valid Common Lisp code.

Macros define "new syntax" kind of like special forms deviate from function evaluation. (if a b c) is a special form, that is a non-function-call defined by Lisp, just like Haskell leverages lazy evaluation to define "if" without eager function evaluation.

[1] http://www-formal.stanford.edu/jmc/recursive.pdf

Reply on news.ycombinator.com