This is why Lisp never catches on. Each programmer invents their own ad-hoc, undocumented, barely working language in the form of those macros. The same thing happens in other languages with macros (like C and assembler).
That's the downfall I am referring to. The trouble happens when the creator of the nifty undocumented kludge language hands it off to someone else, who scraps it and re-implements it in Python or whatever.
I remember when C++ experts discovered "expression templates", which provided the ability to turn ordinary C++ code into a magical kludge language that did not at all behave like C++. It was all the rage for a year or two, then sank without a trace.
I say this as somebody that loves CL dearly. The difference between princ, prin1 and print; set, setf and setq; =, eq, eql, equal and equalp is more than most programmers can be bothered to memorize.
Any sufficiently complicated Common Lisp program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.
I thought it was gross, but then I met Microsoft COM.
"Undocumented" is quite a bit of exaggeration, if not straight fabrication. Lispers historically have been the most adamant docstring writers among all programmers.
Emacs is the best example of self-documented system. Pretty much everything is documented. You can lookup the documentation for every single key, button or mouse movement you perform. You can't ever hope to publish a package if a single docstring is not up to the standards.
> This is why Lisp never catches on
Well, let's see: Clojure, Clojurescript, ClojureDart, Coalton, Janet, Fennel, Jank, Jolt, Joker, Squint, etc., etc., and the list goes on and on. Lisp is the only language that's been practically mainstream for seventy years and refuses to die. I don't think it ever will. Because it's not a concrete language implementation, it's an abstract idea. And an enormously practical idea. Just because someone doesn't get it, it doesn't make it less practical.
Perhaps I wasn't clear. The problem with each developer developing their own custom language to implement their project is someone new to it has to understand both their unique language as well as their project. This is a high barrier to adoption.
Documenting a language properly takes a great deal of effort. It's no surprise that someone inventing a custome language just for their project does not create more than minimal documentation, if any.
P.S. I was writing elisp macros back in the 1980s.
>[…]The same thing happens in other languages with macros (like C and assembler).
Uh, C definitely caught on.
I have seen endless incomprehensible preprocessor crap. People have even used it for metaprogramming.
I used to use it for metaprogramming, too. One day, I decided to rip it all out and use the preprocessor as little as possible. The result was much better.
Analogous to C are the macro processors for assemblers. A friend of mine who worked at Microsoft told me about an assembler program that was about 50k. The person who wrote it had left the company. There was a bug in the program, and the manager assigned it to programmer after programmer, and they'd all give up on it after a couple weeks. So my friend said he'd fix it, and fixed it in a couple hours and checked it in.
The manager was surprised. He asked how my friend managed it. My friend said that the code was all macros to create a magical special undocumented half-assed language that nobody could understand. So he ran the object file through a disassembler (mine) to generate source code. Then the bug was obvious, he fixed it, and checked in the disassembly as the new source.
I've also heard complaints from Scala programmers about the overuse of macros making for incomprehensible code.