At the very root, Python syntax is nearly identical to Java, C, etc.
The syntax of Haskell, OCaml, etc, is a deviation from that of Python, Java, etc, but not an incredibly significant one. Perhaps you may find that the syntax of these languages scares you but you think you could probably learn it after a bit of mind-wrangling.
Lisp, on the other hand, is old. It was developed over 10 years before even C came to being! So the syntax is completely different, and requires you to actually learn it instead of just assuming that your knowledge of other programming languages will pull you through.
In summary, it's not a fault in you. It's just that you are used to C-derived and influenced syntaxes, but not to Lisp; and the syntax of Lisp is different enough that you need to actually learn it.
Here's a crash course:
(f a b c) is the same as f(a,b,c).
'(a b c) is kind of like ["a", "b", "c"].
'(1 2 3) is just like [1,2,3]
(defun x (a b c ...) ...) defines a function x(a,b,c).
(+ 1 2 3) is just like +(1,2,3) = 1 + 2 + 3
(setf x 5) is just like x = 5
'(a b ,c) is just like ["a", "b", c]
if L is equal to '(1 2 3), then
`(10 20 ,@L) is like [10, 20, 1, 2, 3]
By the way, what you might find interesting is that I once showed this to a friend of mine who knew nothing about programming. I showed him how to write a basic program in Java (with a few subroutines and a loop), then how to do it in Lisp. He exclaimed, "Argh! If I can do it like this, then why do I need all those fancy weird words like public and void and what not and braces and... eugh!"
Sorry for that longish post, I hope you found it at least a bit informative. ;)
Until you start getting into the most complex (and powerful) aspects of lisp, the last two lines of syntax are rarely used. You get get a very long way without understanding them, but once you do lisp will become one of your favorite languages.
(let ((prod (apply #'* probs))) (/ prod (+ prod (apply #'* (mapcar #'(lambda (x) (- 1 x)) probs)))))
It took me several minutes to weed through this code. Part of that is because I didn't know what, say, apply did, and had to look it up. That's not an issue because you have to do that when learning any new language. But I had to carefully inspect the source bottom-up and break it apart because I couldn't tell what the arguments are on each function call. An IDE that does block highlighting would help a lot, but even then it feels overwhelming, because it only helps me inspect the block that my mouse cursor is currently on.
I guess what I'm saying is, I feel as if the syntactical constructs in other languages are necessary for me to be able to quickly parse a language.
I let Emacs indent that, and it came out like so:
(let ((prod (apply #'* probs)))
(/ prod
(+ prod (apply #'*
(mapcar #'(lambda (x)
(- 1 x))
probs)))))
Prefix notation takes some getting used to for arithmetic, but that's neither here nor there. You probably have a bundle of idiosyncratic operator precedence rules memorized (quick, does 3*4+5 equal 27 or 17?), and Lisp doesn't use those, preferring to consistently use parenthetical nesting for everything.
See also: http://img264.imageshack.us/img264/1397/lispnd7.png
That example you linked to is confusing because: 1) You don't know the syntax to read it out like a formula, and 2) he uses the names prod and probs which look the same and strain the mind of a newbie.
The way you read "hard" Lisp code, specially one in Graham's style is to read from the last line upwards. The tell-tale is the deep indentation, btw.
Start at the last line, which is the body of MAPCAR, you know mapcar is MAP which applies its first argument function to successive elements of its second-argument, which should be a list.
(mapcar #'(lambda (x)
(- 1 x)) probs)
(defun normalize-probabilities (probabilities)
(mapcar (lambda (x)
(- 1 x)) probabilities))
Look up further and notice the result of this function call, a list, is being reduced to an integer. APPLY applies its first argument, a function, to its second argument, a list. For example, (apply #'+ (list 1 2 3 4)) means sum of the first four digits. C programmers should find this familiar: (apply #'main argv)
Where MAPCAR returns a list, apply returns whatever its function argument is supposed to return. We know the function #'* takes a list of numbers and returns their product. So, the snippet is just multiplying the list of probabilities, or reducing to a scalar.
(apply #'* (normalize-probabilities probs))
(+ prod (apply #'* (normalize-probabilities probs)))
Look up a bit and see where that's coming from. The nearest binding form for prod is in the LET body on top. Looks very similar to something we have seen before. Another application of multiplication to the list named probs. This is not a Lisp issue, it's a Bayesian statistics or perhaps a PG issues. You continue reading the paper and discover his formula in English prose. Aha! :-)
You can move code around, tuck it into functions and do whatever you want with it. See if you recognize the code snippet now:
(defun normalize-probabilities (probabilities)
(mapcar (lambda (x)
(- 1 x)) probabilities))
(defun pg-foo-fomula (product)
(/ product (+ product
(apply #'* (normalize-probabilities probs)))))
(defun main ()
(loop
(let ((probabilities (eval (read))))
(when (listp probabilities)
(print (pg-foo-formula (apply #'* probabilities))))
(princ "OK > "))))
The raw basic "syntax" in terms of the parens is a bear to read (and write), but you get used to it. It may be different, but that isn't really the problem. Anyone can figure out nested parentheses and escaping. Just as someone else pointed out -- this isn't alien territory .. nested parens/brackets/whatever and literal escaping are common in complex/terse code in any of the popular languages. Possibly Haskell is different, but I find it's terse syntax of juxtaposition more inscrutable than lisp, for the most part.
The problem is that in practical terms, the lisp syntax is not just nested parens and various forms of quoting and basic function invocation. The syntax includes the positions of arguments to various standard macros and special forms. Are they are sublists or inline? what are the keywords? And most importantly, are they evaluated or not? From an academic point of few, this technically could be considered semantics of a programming language, but I think that's false -- we don't consider human sentence structure semantics. This is totally syntax. And in lisp, it's highly irregular from form to form. It's just something you have to "know" based on the form, and the lack of syntax other than parens and quotes certainly doesn't help you at all here.
Compared to something like python, it's unreadable and confusing.
However, other Lisps exist. Take a look at Scheme or Clojure. Scheme is prettier, and Clojure more practical.
(On a side note: these different variants of Lisp certainly don't help create one "Lisp community".)
As I see it, the biggest hurdle to learning LISP isn't the syntax per se. Lisp's syntax is so simple, it's easily learned in a matter of minutes. The whole language basically boils down to lists and dotted pairs, and lists can be boiled down to dotted pairs if you squint at them the right way.
The real obstacle for newcomers is that you have to completely rewire your brain. The vast majority of us were taught programming with BASIC, C, Java, or something similar. With those languages, programming is like following a recipe. Do step a, then do step b, etc. Moreover, imperative languages use a familiar "verb" and "noun" paradigm. That's natural and intuitive to most of us.
With lisp, everything gets turned on its head. Everything happens at once. Code is data. There's no syntactic sugar because there's no syntax. You have to completely relearn how to look at computers and programming, and start over as a child. It's incredibly rewarding once you start having those "aha!" moments, but there is a decent-sized conceptual hurdle for people that are used to procedural programming languages.
It seems that the majority of the lisp community suffers from the curse of knowledge. For experienced lisp people, lisp is so natural and intuitive that it's hard to understand why other people have such trouble with it. The OP is correct, lisp could really use more resources to bridge the gap between lispers and the outside world.
I don't have any problem learning the syntax - I didn't have anything against ml-s, haskell and others either, even though I started with pascal. But what do I gain by using lisp seriously? I can do easy prototyping in python; safe, functional code in haskell; serious multiprocessing in erlang; fast tight-loops in c; nice oo in either python, c# or java, etc. etc. Why exactly should I spend more time with lisp? What does it help me achieve? Python right now is basically lisp + community from what I can see. Sytax doesn't even matter that much.
If I need to rewire my brain to feel natural with a language that doesn't give me any new possibilities... then what's the point? I need those wires to understand haskell, sorry ;) I just need some balance - I can change the way I write my code, but tell me what do I get in exchange... and then tell me how is it better than doing the same in any other language.
The syntax also helps with code generation.
Because I can do easy prototyping in lisp; safe, functional code in lisp; serious multiprocessing in lisp; fast tight-loops in c; nice oo in either lisp, lisp or lisp.
If you find a problem that needs a new paradigm, you either need a new language, or you need lisp.
> Python right now is basically lisp + community from what I can see.
Python is a great language. It supports a lot of the most useful and powerful paradigms that lispers use, so many lispers like it and use it.
But python is not "basically lisp", simply because it lacks the property I mentioned earlier.
What do you mean by serious multiprocessing in lisp? Does it have a distributed database included? Or a node registration and message passing framework? Or a process control framework? Can it run my 10000 processes? Does it have portable networking and thread implementation where nothing ever blocks... Or did you mean that I can have for example the sbcl threading http://www.sbcl.org/manual/Threading.html and can build all of that on my own? I could do the same in c... but I won't - erlang is better for that.
I'd also argue with "nice oo in lisp"... for some values of "nice" :) For big projects I want a code browser / ide. I want types in my OO. I want the code to be structured in a way that makes the methods belong to classes. I don't want old-style message passing OO (not saying that it's bad... it's just not what is used these days in big projects). I want new-style enterprise-level OO with interfaces and self-commenting code where you write what you want to do, because every single person in your 100-person team has to understand it without any comments when you're not around.
So everything that's left imho is easy prototyping. We can shoehorn every paradigm into any language - sure. But then you get something that can do everything and isn't good for anything in particular. If I learned lisp before python, I'd probably say the same thing about python... it's just not worth the additional attention, because they're both easy-prototyping languages, where you can promote the code into production when your first design is "good enough".
The real problem I see with Lisp is that it lacks a vibrant community like Ruby and Python has, and is in some cases quite ugly. (Why, why use names like 'progn'? Why not the shorter, clearer 'do'? etc.) Clojure addresses that, but I get the feeling that Clojure is just an addon to Java and that you must use the Java lib in order to use Clojure; the integration between them seems to be to be a bit ugly, if easy to use.
However, due to the size of the JVM base, Clojure is very good for production code - but strikes me as lacking the hack-hack quality you can get with CL. And only has one implementation. Aurgh! If only they didn't standardize CL. (Except, then it wouldn't exist at all right now, I bet.)
That doesn't necessarily follow, but it might very well be true.
In addition to features, languages need speakers. Java is rarely regarded for its feature set, but it is certainly a "good" language in that it is good at something.
The rest of your comment is largely drivel, compounded by the fact you simply don't know lisp well enough to think in lisp, and consequently have a meaningful conversation about what that means. Nevertheless, you somehow got the impression that I thought you should switch to lisp, and I sincerely apologize for that; I never meant that at all- only to express how knowing lisp is valuable.
However, there was one thing in particular:
> I don't want old-style message passing OO
Most lisp programmers prefer CLOS-style OO which uses generic functions, and not message passing.
They aren't necessarily incorrect, they're just meaningless: I don't start a program and say what this program needs is static type safety, multiprocessing and object oriented designs. I start at a much higher level than that: What should the program ultimately do?
A number of years ago, I made survey kiosks that ran on OpenMCL on G3 macs. They had to ask some prearranged questions, with some potential branching and "please specify" type answers, and they could give different questions based on previous answers. On the way, a number of interesting things happened:
1. The original plan was to use xulrunner and aserve on top of an X server (no WM). xulrunner was simply too slow, so with only two weeks left it was ripped out and replaced with checkboxes, buttons, images, "textareas" and radio buttons on SDL. All of the generic functions which used to emit html, now tracked dynamic variables for a "cursor" and made a drawlist.
2. The survey stations updated their code using multicast pings. Basically they had a directory of source files, and a version tag that would automatically be pushed to any nearby station. When I decided I needed a database, I simply stored the answers in that directory, using the mac address of the ethernet device as the file "name", and the contents simply as lisp forms. The result was that all the answers collected by a machine were shared with every other nearby station. On two occasions a machine died in the field, but no questions were lost. They also shared their source code similarly, so I could push out updates very easily.
3. I was out of the state at a wedding on their first deployment, and there was a problem: One of the machines was acting really funky. Using two cell phones, emacs and slime, I was able to connect into the machine remotely and repair it on the live system. Once it was repaired, I saved the file and it automatically copied all over the campus.
4. One of the major focuses was media awareness. The user was asked to enter in all of the advertisements they were aware of. This was completely freeform, but we needed to collect the results, correct misspellings, cluster them, and make graphs in real time as the event proceeded. These reports were used in justification to the event promoters.
5. At our second install, they had no ethernet deployment. Instead they promised someone could walk a USB key between the machines. A udev rule to detect insert copied all the database files from that synchronization directory to the usb stick, and a trigger replaced any local files if they were newer.
From prototype to delivery was about two months, and we used multiprocessing, a distributed system, and object oriented programming.
Could it have been done in Python? Sure, but had I used python I probably would've used one of the web frameworks, and once I decided I needed to draw the screen myself, things would've gotten pretty messy.
Could it have been done in Erlang? Sure (I can use the process dictionary for dynamic variables), but when I build the database synchronization system using Erlang primitives, I lose the ability to add the easy USB support.
Could it have been done in Haskell? Absolutely, but only if I got it right the first time: I wouldn't have been able to update the code while I was at a wedding.
Ultimately, had I a clear idea of what was needed before I started, any one of those languages could've been made to work, and they all have lots of library code I could've lifted, which is your point.
However my point is that I didn't have a clear idea what was needed, and replacing the libraries would've required either a significant shift in complexity, or a paradigm that the "best" language simply didn't have.
I don't always use lisp, because the practical access to libraries and efforts is important, however there are times where not having access to libraries and efforts is more important: When I don't know what a program will need, locking myself into a module or a framework is the last thing I want to do.
Edit: Gruseom is right, I was very critical without providing any support. Lisp is still very useful in certain cases. The instances which I find trouble with it are detailed in my second response.
That would explain why there aren't any Lisp web servers.
It is worthless for prototyping as its modules haven't been up to date in 20 years.
Tell that to the people who are using it. Oh wait, I'm one of them. What you're saying here is such a crock that I have to wonder what your purpose is.
As to multithreading, first since when is a web server the be-all-end-all of multiprocessing? Hell, you don't even need multithreading to do a webserver, just use Unix processes. My issue with Lisp is that there is no standard for multi-threaded Lisp; the ANSI spec doesn't consider threading at all. The Lisp community seems to be increasingly fractured over the years, which is too bad because I'm not insulting Lisp because I hate it -- it has just become too difficult to use as my generic do-everything language.
Your ID3 tag library is a good example of the kind of thing one doesn't find much of in the CL world.
On multithreading, though, I think you're off base. Modern CLs all support it, it works great, end of story. That it's not in the standard is a red herring; you simply look up what you need in the documentation for your Lisp instead of in the Hyperspec. Big deal. If you want to run on more than one Lisp, it's trivial:
(defun spawn (name fn)
#+ccl (ccl:process-run-function name fn)
#+lispworks (mp:process-run-function name nil fn)
#+sbcl (sb-thread:make-thread fn :name name))
Yes. You can practice in python reading this:
[f,b,(),q,[1,7,q],[q],[q,r,6,1]]
Lisp "code" is just a data structure (largely: nested lists). It's not the syntax that bothers you, it's dealing with complex data structures. You can see for yourself that they bother you no matter the actual syntax because the above form is python, and it bothers you as well.
People who seem to pick up lisp quickly, simply have a better- rather, a more compatible background. If they lack your particular impetus, they might already be comfortable with complex data structures.
Yes: People can be predisposed to "getting" lisp, but I doubt it is a genetic thing.
> it takes so much effort for me to figure out what's going on
There's nothing going on. In python, there's an invisible stratum between the code you write and the code the computer executes. You don't usually see the substrate, and if you aren't paying real close attention, you might often forget that it is there. Lisp simply lacks that invisible layer.
If you can't get past the complex data structures bit, there are other languages that lack that stratum that use very simple data structures: Forth, for example.
However, using the language means the syntax, semantics, std lib, and community. So it's ultimately a very personal choice.