Python supports functions as objects that can be passed around, but that does not make it a FP language. Functional programming is about constraining mutability and declaring what you want, now how. Python does not lend itself to that style of programming.
Guido has spoken out against map/reduce/filter, lambda, and TCO in Python so many people interpret that as him disliking FP. I think it has more to do with Guido's aesthetic and Python's design principles than anything else.
"It's doing kwyjibo(i,v) to all the i in zqmfgb. Now, lets see what v and zqmfgb are."
In clojure, that isn't always the case. An example: recently, I was trying to write an OpenGL app in Clojure. I found the following code on the net (pretend it was written by a team member):
(doto obj
(.glBegin ...)
(.methods ...)
(.glEnd))
(gl-begin-end body) ; this should macro expand to:
(do
(.glBegin ...)
body
(.glEnd))
(doto obj
(.method x)
(.method y))
(clojure.core/let [G__1206 obj]
(.method G__1206 x)
(.method G__1206 y) G__1206)
;macroexpand once
(clojure.core/let [G_1725 obj]
(gl-begin-end obj body))
;macroexpand twice
(clojure.core/let [G_1725 obj]
(do
(.glBegin ...)
G_1725
body
(.glEnd)))
In this case, the problem is me: `(doto obj body)` is idiomatic clojure, and I should have known about it. On the other hand, `(kwyjibo obj obj2 body)` could be very hard to figure out and modify.
I should, however, raise the caveat that I'm not a good enough lisp programmer to know if this is really a problem in practice.
(doto obj
(.glBegin ...)
(.methods ...)
(.glEnd))
(gl-begin-end body) ; this should macro expand to:
(do
(.glBegin ...)
body
(.glEnd))
Here's how you can do this:
(defmacro gl-begin-end [obj & body]
`(doto ~obj
(.glBegin)
~@body
(.glEnd)))In python, I know what this code does, regardless of what happens at higher levels of indentation (ignoring some rarely used class-level hacking):
f(a,b,c)
g.h(i)
(f a b c)
(.h g i)
I'm speculating that this may make collaboration more difficult.
No, actually, they don't. Given the usual high quality of your comments, I'm surprised you would repeat this canard.
you can look at the inside of a function and have some idea what it is doing.
This is just as true of a macro. In fact it's more true: you can expand a macro in place on your code and see what it is doing, a maneuver there is no exact analog to in functionland. The macroexpansions in your example make it immediately apparent what the problem was.
Macros are different from functions. That's why they're useful. I'd encourage you to stick with the learning process. The key difference with what you were expecting is that macros control the evaluation of their arguments.
To understand a line of code, you must understand enclosing sexps before you can understand what the line of code does. In a macro-less language, that is not necessary. It is something extra you need to think about (and I agree that C-x C-m helps), and that makes it harder to reason about.
I never said they aren't useful (and I'd strongly disagree with someone who did). It doesn't change the fact that they can make understanding code harder.
Yes, but I was commenting on your example: it was what the macro was doing to control evaluation that caused it to depart from your function-centric intuition.
As for the general point - that macros "can make understanding code harder" - I haven't heard any argument that doesn't apply just as much to functions mutatis mutandis. A function adds complexity too. You have to understand its name. You have to know what each of its arguments is there for. To truly understand a function, you may have to look at its source code. Therefore functions can make understanding code harder!
Except, of course, that they make it easier, assuming they're used properly to organize a program, and people who have experience with them consider them indispensable. Same for macros. Most of the objections people bring up are exactly this sort of pseudo-argument that, if it were true, would in fact be a reason to avoid higher-level languages.
If you're really making a general claim to the contrary, please come up with more than a single trivial example that is confusing only at the beginning of the learning curve.
A function call (in a lexically scoped) language without side effects can be completely understood only by looking only at the function definition.
A function call with side effects now requires extra knowledge: you need to know the state of the world (or some subset) before it is called. Side effects make a program harder to reason about, and can pose a problem in a team.
Similarly, a function or macro call (ignore side effects) also requires extra knowledge: you need to know not only the definition of your function/macro, but also the definition of all macros enclosing the current one. This also makes it harder to reason about.
So my argument "favors" Haskell over Python and Python over Lisp. It also favors Haskell over Liskell, and Clojure over Elisp (due to lexical vs dynamic scoping).
Regardless, I'm not trying to advocate against lisp use. I'm just making a point that the non-locality of macros can make it harder to reason about, and that could be a problem in a team environment.
it was what the macro was doing to control evaluation that caused it to depart from your function-centric intuition
I took a closer look at the example and now see that it had nothing to do with evaluation. Sorry about that. Clearly, the problem was that doto was transforming (gl-begin-end body) into (gl-begin-end obj body) because of an assumption that the forms passed to doto are always OO method calls.
Yuck! Now I understand your complaint about the enclosing form transforming the enclosed form in an unexpected way. What doto is doing there strikes me as just wack from a Lisp point of view. I can't think offhand of many (any) macros that modify the forms they enclose while still evaluating them. Obviously this is completely non-composable. No kidding it leads to unexpected behavior.
What's going on here doesn't have so much to do with macros in general, but rather an impedance mismatch in Clojure between the Lisp side and the Java side (and perhaps ultimately between FP and OO). It's a non-issue in CL - so much so that I completely missed the point until the third time I read your example. It makes me wonder how many other such mismatches Clojure must inevitably be afflicted with.
I'd still argue, though, that you're drawing a false conclusion about macros in general. The problem with (doto obj (gl-begin-end body)) is very much with the definition of DOTO. It has a weird side-effect, true, but one which is still local: it affects the form it encloses, not one which encloses it. The cognitive burden here is analogous to functionland: in order to understand f(g(x)) you have to understand both f and g. f might do something weird that violates the intention of the author of g, but understanding this is incumbent upon the author of f(g(x)), not the author of g. Of course the real culprit may well be the author of f for writing something prone to unwelcome surprises.
In Lisp and Smalltalk, you can look at the inside of a function and have some idea what it is doing.
With Lisp you can use #'DISASSEMBLE and with any Smalltalk worth its weight, you can simple click around and find the method that you want to look at
But I think something else is going on here, the Inscrutable Macro idea serves as a kind of "intuition pump" (of the negative kind) http://en.wikipedia.org/wiki/Intuition_pump to try and demonstrate macros' badness. If someone isn't very familiar with lisp they'll offer code that's merely not idiomatic, when someone is familiar with lisp then they'll offer up something that's just downright pathological, and I'll be the first to admit, that a pathological macro can be really pathological. At this point, equipped with their executable straw man, they claim "this macro is insane, and so macros are unmanageable."
If the measure for unmanageability is just that impossible-to-understand code can be written in some language then the IOCCC means C shouldn't be used.
In practice the Inscrutable Macro just isn't written.
The problem with this fallacy is that it assumes that what is true of a single self-contained function is true of thousands of inter-depending functions heaped on top of each other.
I find typical Python programming to be not particularly disciplined, so if you've overcome the problem in that language you should be fine in a generally more disciplined language like Clojure.
If you are in this situation, I would suggest a full rewrite with policies that prevent your team from building such a mess.
The real point though (which remains unaddressed) was that the argument against higher-level abstractions is wrong because it assumes that what is true of a single function written at a certain level of abstraction will remain true if you try to build an entire complex system out of them. It won't. Therefore, as complexity grows, we need the ability to create new abstractions. There is something perverse about the argument that we should eschew powerful techniques for doing this because they'll make large systems hard to understand. It's like saying we should avoid water because dehydration is bad.
Some of us employ techniques that make it easier to maintain large chunks of code. I refrain to describe them as complex because complexity is something to be avoided like the plague.
You can think of the current *BSD codebases as direct descendants of the original AT&T UNIX codebase that have survived by employing one or more of the techniques I mentioned in the previous paragraph.
The whole "people don't write applications in Lisp, they use Lisp to write a new language and then write applications in that" point has been brought up already, so I won't try to argue its pros or cons (thankfully somebody else can get flamed from both sides over it).
I will say that I think Python does something interesting, in that it removes some of the stupidest problems teams usually run into. Consider projects written in C or C-like languages: hugely disproportionate amounts of time are dedicated not to actual issues of how to design applications, but to things like where to put braces and how to use whitespace. Python sidesteps a lot of that petty bickering because the requirements of syntactically valid Python impose consistent solutions to those debates from the start.
Also, culturally, Python programming style tends to go quite a bit beyond that. For example, most large projects simply use PEP 8 as their style guide, doing away with an even larger class of stupid infighting.
As to FP and whether Python hates or supports it, unfortunately it is the case that if you put five FP proponents in a room you'll get at least twelve mutually-exclusive definitions of FP. So I think that tends to be a rather silly debate to have :)