Reader Macros in Common Lisp

When Harlequin started out with Dylan, to bootstrap we first quickly morphed LispWorks into a high-functioning Dylan IDE. This was done through a combination of macro, reader macro and CLOS MetaObject Protocol abuse, allowing us to achieve a faithful rendition of Dylan's syntax and object system semantics respectively.

When Dylan went from prefix to infix syntax the reader macro indirection got all the more elaborate: what started out as a trivial reader macro that treated colons differently started delegating to a full-blown lex & yacc style parser instead.

All the better, most of this was achieved modularly: Lisp editor buffers and Lisp listeners would run happily alongside Dylan editor buffers and Dylan listeners, Lisp code could still compile load and run alongside Dylan code, and Lisp classes could interoperate cleanly with Dylan classes.

There's no question that Common Lisp and its predecessors are some of the most formidably adaptable and hackable programming environments ever created. Only Smalltalk really comes close, and they have complementary strengths I think. Standard macros, reader macros and a comprehensive MOP are some of the key things that stand out in Common Lisp's favour in that hackability comparison.

It's things like this that remind me that Lisp(s) are really the best languages for taking advantage of the idea of Turing completeness. In terms of ease of implementation that is. I think that really, this is what we mean by "powerful" languages - how easily can you take advantage of the principle of completeness to do anything in your chosen language.

You often hear people using the idea of completeness to say that their favourite language can, of course, do anything. But in reality some languages just aren't very well suited to it, or would require some horrendous amount of coding and abstractions (Greenspuns Tenth rule comes to mind [1]).

So whenever some new idea in programming becomes popular, whether it's pattern matching [2], concurrency [3], or even the web (HN), you can always rest easy because someone will be able to quite quickly write that functionality into Lisp itself. The only difficulty is in finding out about the libraries in the first place!

[1] https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule [2] https://github.com/m2ym/optima [3] http://lparallel.org/

I often read that Ruby (or Forth, for that matter) is great for DSL because you can "misuse" the language tokenizer/parser directly as parser for your DSL, as long as it's similar enough to ruby (e.g. tokens are whitespace delimited and such). Unfortunately, Ruby's macro power is nowhere near LISP.

I wasn't aware that this is possible in LISP, too. That's really great!

Great article! I've used reader macros to implement readers for little assembly languages. It's also super-awesome that Lisp has this nice compiler right under the floor boards. And I don't have to touch any parser generators.

Love this. Since readmacros aren't an everyday topic but rather out of the way and related to special purposes, there aren't that many practical introductions. This article is a nice writeup.

The author has taken a great care to communicate his ideas clearly in such a way that is a pleasure to read it. Very well done!