Hacker News Reader

Comment by programnature | Hacker News Reader

Pattern matching is the missing feature. I keep thinking about switching to Clojure from Mathematica, but then I think "How can anyone get anything done in Clojure? It doesn't even have pattern matching."

Its not something that can get patched in a library, because the way symbols and evaluation need to work is different (and simpler) than in a Lisp. There is no distinction between macro-s and nonmacros - everything is just a tree transformation.

The main benefit of first-class pattern matching is that your function definitions get a lot more succinct and expressive, since you can encode quite a lot of information in the structure of the arguments, and elegantly unfold the definition from the short and common case to a parameterized sequence of generalizations.

View on HN

What exactly do you mean by pattern matching? I understand it in the ML sense. The Lisp language I use, PLT Scheme, has extensible pattern matching: http://docs.plt-scheme.org/reference/match.html The implementation isn't simple but the techniques are published (see "Pattern matching for Scheme" http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.53.2...) so any Lisp language could implement this. Clojure has less powerful pattern matching, but it does the most common stuff.

The linked article is really about Common Lisp, where the issue is a standard that hasn't been updated in a long time. This doesn't stop individual implementations from making their own advances but in my limited knowledge of the CL implementations this doesn't seem to be occurring.

you might check sometimes the documentation of CL implementations for the advances that they provide over the standard

This is actually occurring. Most advances are made in cross-platform libraries (networking, regexps, threading, pattern matching, lazy evaluation, etc.) but some implementations have their own extra that make them interesting (by no means an exhaustive list):

ABCL (http://common-lisp.net/project/armedbear/) runs on the JVM.

CCL (http://www.clozure.com/clozurecl.html) has excellent integration with Objective-C and Cocoa on OS X.

ECL (http://ecls.sourceforge.net/) is easily embeddable in C / C++.

The commercial offerings Allegro CL (http://www.franz.com/products/allegrocl/) and LispWorks (http://www.lispworks.com/) keep on improving and extending their own implementations in interesting ways.

Good question. Here is what I mean:

1. Pattern matching as the basis for function definition, to determine which code executes and expedite argument destructuring.

2. Patterns themselves should have first-class representation (preferably symbolic), so you can generate them in one place and use them in another.

3. Implicit in this is that the structure of the language is systematic enough to make this worthwhile, meaning something s-expression based, or perhaps something like Scala that achieves similar ends in a much different way.

How close does Qi (http://en.wikipedia.org/wiki/Qi_%28programming_language%29) come to what you desire?

A little bit of googling gave me the sense that pattern matching in Mathematica is different from the pattern matching present in several functional languages. Please let me know how these Clojure features differ from that:

Clojure has pattern matching for the arguments in a function definition similar to ML or Haskell. It's often used as a more verbose alternative to optional args, but it's more powerful fundamentally.

Clojure has destructuring in binding forms like let, described here: http://clojure.org/special_forms

Clojure has multimethods which let you pick the method to use based on an arbitrary dispatch function.

There's a pattern matching macro that seems to be the beginning of a DSL for predicates: http://www.brool.com/index.php/pattern-matching-in-clojure

There is 1 mechanism instead of 4, and it also happens to be the fundamental basis of evaluation.

There is no way to measure "absolute power" of elegance, which is the issue being gotten at here.

Not to say Mathematica has it gotten it perfect either, because it hasn't. But I have yet to see a superior implementation, and have seen a lot of clunky ones in my search.

I certainly miss ML-style pattern matching when I'm in Lisp. I sometimes find myself (poorly) simulating the idea with a whole bunch of multi-method specializers.

Based on my limited experience, it seems like you'd really need a more static type system to make the most of pattern matching though. Are there any dynamically typed languages with powerful/useful pattern matching?

As far as doing it in Lisp goes, this is an interesting attempt: http://common-lisp.net/project/cl-match/doc/clmatch.htm

Prolog

Prolog has unification, which is considerably more powerful than just pattern matching. The semantics of Prolog require the ability to unify data structures with some parts not-yet-specified, and backtrack later if they cannot be bound to something valid. In contrast, pattern matching happens all at once and is unidirectional. (Though even basic pattern matching is tremendously useful, IMHO.)

Erlang's pattern matching is somewhat like Prolog's, but with backtracking removed. (Backtracking clashes with Erlang's other semantics.)

Erlang of course.

Putting structure assumptions into function argument lists is not necessarily a good idea. One exposes the implementation.

Pattern matching function definitions are easy to do in Lisp.

But given the nature of Lisp, much of the pattern matching then needs to be done at runtime - which leads to less efficient code and invites people to write totally inefficient code.

Consider the Mathematica function Take[], which would save millions of man hours if it existed in other languages.

Take[{a,b,c,d},2] --> {a,b}

Take[{a,b,c,d},-2] -> {c,d}

Take[{a,b,c,d},{1,3}] -> {a,b,c}

Take[{a,b,c,d},{1,-1,2}] -> {a,c}

Take[{{a,b,c,d},{1,2,3,4},{5,6,7,8}},2,2] -> {{a, b}, {1, 2}}

etc. In my book this is clearly useful, and its only scratching the surface. ( {} is actually List[] in Mathematica FullForm ... )

The point of symbolic representation is to represent the intended meaning. Its nothing about the "internal" implementation. Using symbols and simple tree structures to compactly express stuff is extremely powerful, and not coincidentally the essential way that human language works.

Sure, with a sufficiently dynamic language you can implement such functionality. The problem with Lisp is that by default symbols want to evaluate, and if you want to treat them symbolically you have to operate in a "special" mode. This makes things too complicated. There is a reason you don't see meaning represented by structure in pretty much any other language besides Mathematica.

It looks like you have never programmed in Lisp.

A function like Take is easy to write in Lisp. Lisp has many similar functions like that - but with a better interface.

> There is a reason you don't see meaning represented by structure in pretty much any other language besides Mathematica.

Could it really be that you missed the AI software that has been written in Lisp in the last five decades?

What is ' other than a special mode? The fact is that symbols are treated more systematically in Mathematica, and that makes it easier to assemble and dissemble symbolic structures of all sorts.

Sounds like a case of blub. You look at Mathematica and see some weird stuff that is probably equivalent in power to multimethods or whatever, I look at Lisp and think how can I possibly live without civilized pattern matching.

As far as Lisp-based AI goes, its in fact very easy to miss it, but this is probably not the thread to get into Lisp's cultural issues.

What is ' other than a special mode?

I usually implement it as a reader macro. :)

it's a shortcut to quote

it's a shortcut to quote

Yep.

My comment was about implementing one's own Lisp reader.

    (define *reader-macros* `(
      (#\' ,(lambda () (list 'quote (read))))))

instead of

    (case dispatch-char
      ((#\') (list 'quote (read)))
      ...)

Symbols are not treated more systematically in Mathematica.

QUOTE is a special operator in Lisp that causes the evaluator to return its argument unevaluated.

     (quote a) -> a
     (quote (+ 1 2)) -> (+ 1 2)
     (quote "abc") -> "abc"

Mathematica has a different strategy for computation, one that is based on rewrite rules. Expressions are rewritten until they can't no longer be rewritten.

In Lisp evaluation

    (+ a a)

gives an error if the variable A has no value.

In Mathematica it would be reduced to

   (* 2 a)  ; in Lisp syntax

if A has no value.

But that has little to do with the processing capabilities of Lisp. A function like Take can be written very natural in Lisp - there is NO special mode needed. The function takes data and computes results.

You may want to get out of your Mathematica blub and learn some Lisp. Writing the 'Take' function is a good exercise. Stuff like that are basic exercises in Lisp courses.

Take is a really terrible example, even python has take built into the syntax (Take[lst, 2] -> lst[0:2]). But Mathematica does have nice pattern matching which few lisps have:

    SolvePoly[a_*x+b_=0] := { -b/a }
    SolvePoly[a_*x^2+b_*x+c_=0] := \
        { (-b + sqrt(b^2-4*a*c))/2a, (-b - sqrt(b^2-4*a*c))/2a }
    SolvePoly[_] := "I only took high school algebra"

Of course, in principle one could write a pattern matching macro in lisp, and I imagine there are already some halfway implementations of it. That's pretty much just Turing Equivalence.

The point was not lst[0:2] in Python. See the definition of Take in Mathematica - it is quite a bit more capable.

You need also differentiate between 'pattern matching' and 'rewrite system'. Pattern matching is just taking a pattern, some data and see if it matches.

    (match '(+ (* ?a x) ?b) '(+ (* 123.0 x) z)) -> T

Routines like the above are found in many books about Lisp and have been provided in Lisp libraries for decades.

Specifying rewrite rules with patterns for mathematical purposes (simplification, integration, differentiation, ...) is also almost as old as Lisp. Norvig's book 'Paradigm's of AI Programming' explains how it is implemented in Lisp. These things are at the heart of several computer algebra systems written in Lisp - like Macsyma.

how would one define SolvePoly or even Take for that matter in terms of Mathematica if it were not "built in"? I am not saying it is impossible (I really do not know and thus I am curious).

Same way you would do it in lisp - you'd build a function to do pattern matching and use conditionals to test which pattern is matched.

    (defun solve-poly (p)
        (if (match '(+ (* ?a x) ?b) poly))
            (-b/a  where b, a come from (extract-values-from-pattern-match poly))
           (some code for second order)
     ))

(Some code borrowed from lispm's comment. Not sure which library the match function comes from.)

Clojure and Scheme have take and drop, though the semantics are slightly different from those of Mathematica. They can be written in almost any language.

My point is that putting "structure assumptions in argument lists", though not a good idea 100% of the time, is often extremely useful, as illustrated by the example.

The greater point is that if this is so, one might as well deal in a model of computation that is a natural fit for that way of thinking.

I've been doing a lot of work in Scala and Actionscript. How I wish they had that family of functions in full generality. Why are they missing from most languages? Because that is not how people think in them.

Your example above did not show any special pattern matching capabilities whatsoever.

> There is a reason you don't see meaning represented by structure in pretty much any other language besides Mathematica.

Actually, this is done in Lisp and Prolog all the time.

Off the top of my head: J, K, Haskell, and Joy have take functions, and writing them in Lisp and Prolog is not difficult.

Wow, Lisp has Take. I wrote one. It took me a few minutes:

    CL-USER 58 > (take '(a b c d) 2)
    (A B)

    CL-USER 59 > (take '(a b c d) -2)
    (C D)

    CL-USER 60 > (take '(a b c d) '(0 3))
    (A B C)

    CL-USER 61 > (take '(a b c d) '(0 -1 2))
    (A C)

    CL-USER 62 > (take '((a b c d) (1 2 3 4) (5 6 7 8)) 2 2)
    ((A B) (1 2))

That was easy.

would you mind posting your definition?

    (defun take-1 (it what)
      (cond ((eq what :all) it)
            ((eq what :none) nil)
            ((and (numberp what) (plusp what))
             (subseq it 0 what))
            ((and (numberp what) (minusp what))
             (last it (- what)))
            ((and (consp what)
                  (= (length what) 1)
                  (numberp (first what)))
             (nth (first what) it))
            ((and (consp what)
                  (= (length what) 2)
                  (numberp (first what))              
                  (numberp (second what)))
             (let ((end (if (minusp (second what))
                            (+ (length it) (second what))
                          (second what))))
               (subseq it (first what) end)))
            ((and (consp what)
                  (= (length what) 3)
                  (numberp (first what))
                  (numberp (second what))
                  (numberp (third what)))
             (let ((end (if (minusp (second what))
                            (+ (length it) (second what))
                          (second what))))
               (loop for e = (subseq it (first what)) then (nthcdr (third what) it)
                     for i from (first what) below end by (third what)
                     collect (first e))))))


    (defun take (thing &rest description)
      (cond ((null description) nil)
            ((and (consp description)
                  (= (length description) 1))
             (take-1 thing (first description)))
            (t (loop for e in (take-1 thing (first description))
                     collect (apply #'take e (rest description))))))

> The problem with Lisp is that by default symbols want to evaluate, and if you want to treat them symbolically you have to operate in a "special" mode.

So you're saying that variable bindings should be lazy? That if a symbol is not bound to a value, it should be treated as an unevaluated symbol?

I'm not quite certain what your point is. The actual functionality of Take can be easily replicated in most languages, and indeed, most languages have a version of Take called "slice".

But if your example is meant to demonstrate why lazy symbol evaluation is good, then I'm afraid it fails on that point too. There doesn't seem to be anything in your example that is particularly noteworthy.

Could you perhaps explain your point in more detail?

If you don't know how to write a pattern matching library in Lisp, you don't really know Lisp and should probably refrain from spreading misinformation about it.

Reply on news.ycombinator.com