When I die, my first question to god will be 'sir, why the hate on lisp?'. I expect the answer to involve testing of some kind.
Or maybe it's just the historical artifact that everybody claims to be, and if the PDP-10 supported a lisp environment, things would be different. But if I had to bet, I'd bet on metaprogramming not scaling.
For that matter C++ templates are metaprogramming from what I understand, and C macros are a really really primitive form of metaprogramming. Java also has reflection which is metaprogramming-ish.
I don't think the problem was metaprogramming.
For C++ templates, you get that impression and multiply by a few googols.
Now, Ruby and Python communities do claim to gain power from metaprogramming. I can say that Python metaprogramming is something completely different from Lisp's one, but Ruby's is more similar. Yet, both communities put a hard limit on the amount of "magic" you should use on your programs (where Ruby's limit is way higher than Python's), so it can not become the panacea that it is in Lisp.
Metaprogramming is one of the main selling points of Lisp, the only other being how easy it is to implement. Other languages have other features to show.
Macros and the repl were, to me, the best part of clojure.
http://www.winestockwebdesign.com/Essays/Lisp_Curse.html
That basically every program written in a Lisp becomes its own programming language. It has mechanisms which are vaguely-familiar-but-different from all the other programs. The mechanisms are 80% of a complete solution, but a different 80% each time, depending on the needs of the program.
Some humans can write Lisp. But Lisp production code can only be maintained and updated by AIs who are more intelligent than we are.
(http://ithare.com/cpp-guidelines-made-to-measure-s-one-size-...)
Each project has its own coding guide and idiosyncrasies. That is not restricted to Lisp. Macros allow you to express some rules in a domain-specific language, not in a separate document or tacit knowledge. This is more manageable than ad-hoc approaches. Abusing macros is definitely bad and can lead to ghetto-languages.
Lisp is interactive, call "describe" on whatever you don't understand. Under Emacs, if you encounter a form you don't understand, point to it, "C-c C-d d" and you get the documentation; "M-." and you go to the definition; "C-c M-m" and you call macroexpand.
In effect, it's not really scaling; it's being limited to the small portion of developers who work on compilers.
I wonder if there really are mental types well-suited to Lisp and types ill-suited to it.