Can you expand on this. What is something that macros can do that decorators can't? And likewise, what are some common uses of macros that decorators can do?
That said, the usual answers to the question of "Why do I need macros" are slowly but surely being chewed through by Python. The relatively-recent (albeit years old) addition of "with" took another previously-macro-only use case away. I'm not sure what massive win for macros is left. If your Python is "downright verbose" you're probably doing something wrong. It may not always be the Absolute Shortest (TM) but "downright verbose" shouldn't come up often.
From my point of view, not being able to define new syntax is as serious a limitation as not being able to define new functions, or data structures.
There are a number of specific examples usually cited to explain why the ability to define new syntax is important. Answering these specific examples by adding more static syntax to the language seems to be severely missing the point, to me.
Imagine it was functions instead. "Well your language already has a sort() function, but say you want to use an in-place sort instead of what you get with the standard library? You need the ability to define your own functions." "No we don't, our BDFL added an in_place_sort() routine to the standard library for the next version of the language." That doesn't make sense to me at all.
There's a pretty big difference. Both are about DRY. But functions are about abstracting functionality. Macros are about abstracting syntax. Both are useful and can reduce repitition, but in the general dev community, I don't think its clear that everyone is bought into abstracting syntax too much.
For example, the far more mundane operator overloading debates were about just the topic and the side arguing that not all syntax should be abstracted largely won. This is one reason that despite the fact that Lisp is one of the oldest programming languages and easiest to learn, it has never caught on in a big way. It tends to optimize for that which a lot of people don't want optimization.
This seems like you're saying "python is better because it does the things people like and doesn't do the things they don't like". But the same argument could be used to claim Microsoft Windows is a great operating system, and that Justin Bieber is a great musician.
> Lisp [...] has never caught on in a big way [...]
This is factually incorrect. Scaled for the size of the industry, it was at one time as popular as Java was a few years ago. What happened then is well documented elsewhere. Search for "ai winter" and "worse is better" to start.
This is factually incorrect. Scaled for the size of the industry,
Sure, if you scale for the size of the industry which found it useful. But that seems nearly tautological.
I mean the size of the industry that programmed computers at all, not the size of the industry that programmed in Lisp.
Which is exactly my point. We've gone from "Well, I can use it to define a ternary operator" or "I can use it to define a 'with' block" or "I can create continuations" or a number of other specific examples to "Uh, well, I just like them, here I do this with them" and in my experience there's always an easy "Well, the Pythonic way to do that is this and it's not all that much harder, may even be easier, and is a standard part of a relatively simple language rather than your own hand-rolled thing".
It's a very different argument than it was ten years ago when I first got into Python. It's gone from really strong pro-macro points to personal opinion. I respect and acknowledge that opinion, no sarcasm. But it's not the same.
Your last paragraph isn't relevant to me because the specificity of your example is misleading. It's really hard to come up with an actual macro use case that is not covered by modern Python and is still actually a good idea. (It's trivial to come up with bad uses of macros not covered by Python, but I'm hardly crying about that.)
sqlalchemy (and now mongoalchemy) is begging for macros. See CLSQL for an answer.
Regular expressions are infinitely more readable as expressions in a dynamic syntax. See the rx package in Emacs for starters.
What if you want shell/Ruby style back quotes, which in essence create a closure that runs a batch job? See my SHELLSHOCK library (jfm3.org) for the Common Lisp extension. It's not much code at all.
What if you want large literal strings which respect the indentation of their surrounding code, without uglying things up with <<<EOF ? See my BOXEN library (also jfm3.org) for that Common Lisp extension.
What if you want something like Python's r"string", but you want more control over which things are escaped and which are not? See CL-INTERPOL for that.
What if you want anaphora in your control structures?
What if you want Lex/Yacc style parser generator specifications where the type of productions is extensible, and the productions themselves can be generated dynamically at compile time? (This will make no sense to you unless you've ever had no choice but to run m4 on your .l and .y files.)
I could go on. It's not hard for me.
What you can do though is define behavior that varies based on the context. For example in Python you don't have function overloading or types, but with a decorator you can effectively create that type of behavior. So while you're not changing the language, you are changing the semantics of the functions you decorate.