Problems with Lisp | Hacker News Reader

Problems with Lisp

blog.jrock.us

23 pointsjawngee37 comments

nickb posted this article a couple of hours earlier:

http://news.ycombinator.com/item?id=449240

The duplicate wasn't detected even though the urls, after submission, are identical. It turns out there's a bug (a corner case, really) in how duplicates are detected. Probably nickb put this in the Submit field:

  http://blog.jrock.us/articles/Problems with Lisp.pod

while jawngee used this:

  http://blog.jrock.us/articles/Problems%20with%20Lisp.pod

If you resubmit with the first string you get taken to nickb's submission, while the second takes you to jawngee's.

On another note, how many μStallmans of insensitivity would it take to notify pg of this triviality the day after the birth of his first child?

μStallmans? I love it. Your concept, or did you steal it?

Alan Kay famously said "Arrogance in computer science is measured in nanodijkstras." (http://www.youtube.com/watch?v=s7ROTJKkhuI). To which someone wittily answered, "And microkays".

It's worth noting that Clojure has native support for hashes via the {} syntax. (http://clojure.org)

And hashtable lookups are possible by just function-calling the table itself with the key:

  => ({ :foo 1  :bar 2 } :foo)
  1

the common pattern of defaulting to a value if the key isn't found is easy too:

  => ({ :foo 1  :bar 2 } :baz 4)
  4

The above is a hash map, #{ ... } creates a hash set which otherwise has similar semantics. Sets, being function-callable, can be used directly for checking if something is one of a bunch of values:

  (if (#{301 302 303 307} response-code)
    ; we've been redirected

Maps and sets (you also have sorted maps and sets and a compact struct-map, all of which have identical semantics as the hash variants where appropriate) can additionally be used with most functions accepting a seq (list) or can be converted to a list explicitly using the seq function.

Basically, maps and sets really are first-class citizens in clojure. Some of the flexibility is down to the fact that Clojure is a Lisp-1: these features are therefore hard to replicate in CL. (and yes, hash tables also bothered me somewhat in CL, altough other things probably bothered me more)

Something I find strange about hashes in Clojure, is that you can access them using the key as a function. So instead of

    ({ :foo 1  :bar 2 } :foo) ;; => 1

You get the equivalent by doing

    (:foo { :foo 1  :bar 2 }) ;; => 1

For default values:

    (:baz { :foo 1  :bar 2 } 3) ;; => 3

I vaguely remember this is because keys act as functions, and hashes act as functions as well. Out of habit and similarity to other languages, I never use the key-first notation, nor can I see where it is better to use it.

it's likely for usage in higher-order functions. i haven't used clojure but i assume you can do something like this:

  (map :foo hashofhashes)

which would get the element :foo of every hash in hashofhashes

That's another good point, although the alternative isn't much longer:

  (map #(% :foo) hashofhashes)

or if % has a chance of being nil:

  (map #(get % :foo) hashofhashes)

Yep, the funcall semantics of symbols and keywords are to forward the symbol and key itself along with the arguments to the get function.

I do sometimes use this notation, mostly when:

- the map could be nil. (get nil :foo) and (:foo nil) will return nil, but (nil :foo) will throw an exception.

- dealing with struct-maps, where the key can effectively be seen as an accessor function. (cf structs in CL)

- (rarely) when the expression evaluating to the map is long and complex. A keyword in the function position (a) makes it clear that what follows must be a map or set and (b) you don't have to start counting parens.

Clojure definitely has nice data structures.

Does anyone do this sort of hash-table-as-function thing in Scheme?

My article was actually a response to all of the "CL sucks, I'm switching to Clojure" articles I've read this week.

While Clojure is certainly a good language, I think people are "switching" to it from CL due to hype, rather than their deep understanding of the problems with CL. Clojure gives you a few nice features, like immutable data structures, "nicer" reader syntax, and JVM compatibility. In exchange, you give up CLOS in favor of Java's broken object model.

All I'm saying is that I don't think the popular community is really considering CL's advantages before switching, they are only considering the disadvantages. If you are going to consider disadvantages, it's important to keep in mind that Clojure has them too.

(Another often-cited reason for switching to Clojure is Java's libraries. In my experience working with Java, its libraries are pretty bad. Read the URLConnection class that comes with Java 6. It is clear that the API designer has no idea what OOP is, and it's clear that those implementing the library had no idea how to program. CL libraries, I've found, are usually better-done.)

But with that in mind, I really like Clojure. I have the occasional client that insists on JVM applications, and Clojure is significantly less painful than pure Java (or JRuby, or Scala). But it is not my first choice when the JVM is not a requirement.

Clojure provides several ways of interfacing with Java's object model, but generally this is just for compatibility purposes. The equivalent to CLOS in Clojure would be multimethods and hierarchies.

Title should be "Perceived problems with hash-tables in Common Lisp"...

It's true that it would be interesting to have native syntax for hash-tables so you could read and dump them like you can lists and arrays and other stuff. Surprisingly the article doesn't even mention that which is the only real problem with CL's hashtables, aside from the inability to define arbitrary lookup mechanisms beyond eq, eql, equal and equalp...

Having such a cumbersome syntax for hash tables is just as real a problem as the verbosity some Lisp programmers hold against e.g. Java.

One solution could be a read macro that translates into the (setf (gethash ...)) form. Lisp read macros are usually prefix, though, and [key]hash is awkward. OTOH, adding suffix read macros (maybe only of the form symbol[... values], e.g. hash[# key] or hash[: key], with brackets and char(s) at the beginning to represent which suffix read macro.

Or, making the hash a function: (hashname key) to get, (hashname key newval) to set.

> Title should be "Perceived problems with hash-tables in Common Lisp"...

Good idea. Title changed :)

I decided not to bring up the printed representation, since other languages don't do much better. Perl will print either the keys and values with no spaces between them (by default), or something like "5/8" in scalar context, or "HASH(0x123456)" if it's a reference. No better than CL, and nobody complains about this :)

Lisp has support for {} syntax for hashes. It's just not built in. Use a reader macro.

The problem is not the complexity of the api but how verbose it is. Having to type

    (setf (gethash "foo" *hash*) "OH HAI")

just for something as trivial as "hash['foo'] = "OH HAI"" is really too much work and a significant disincentive to using hash tables.

Setf is the generic form (and it's pretty cool how setf works). There are more succinct ways of putting it.

But you need to remember that functional languages discourage that kind of mutable state. It should be no surprise that some things (like assignment) are more awkward in that environment. It's a tradeoff.

Yes but lisp is not a functional language and (to my incomplete knowledge of CL) setf is the only way to modify the hash table. Thus, people have to use that form if they want to use hash tables.

In fairness you could define a function that did it for you:

  (defun assoc (table key val)
    (setf (gethash key table) val)
    table)

or something like that. (this is from memory, hope it's right) You can then let it take a variable number of arguments to make it even more concise. (yes, I've stolen the assoc semantics from Clojure, minus the pure functional aspect)

Well, a macro but yes. Still, if you have to write a macro for something as trivial as that, that in itself is a problem. Especially when other people start reading your code.

Why would you need a macro? Surely a function will suffice.

Whether or not this is acceptable is a matter of taste I guess. I couldn't get on with CL either and was glad to find Clojure, which is now my language of choice.

And especially since people just learning the language aren't going to know how yet.

Last project I worked on had 9154 lines of lisp code out of which 4800 lines are utilities, that I've accumulated and written.

The utilities cover everything from hash, list, string, files, function, a simple version of prolog, some algorithms like bloomfilter, suffix-tree, a simple bayesian filter implementation and a lot of stuff makes writing code easier.

So it's pretty normal to have a large set of utilities that you prefer. It sort of ensures that the code doesn't become to large.

I use (hput hash key value) and (hget hash key) a couple of functions, which I have. So it's a bit verbose than hash[key] = value or hash[key]. But it lets me write stuff like...

(defexample cineworld-cinema-page

  (:record :cinema-listing

    (:name "ENFIELD")

    (:record-loop :film

     (:name "BEVERLY HILLS CHIHUAHUA" "BRIDE WARS")

     (:director "Raja Gosnell" "Gary Winick")

     (:starring "Andy Garcia, Drew Barrymore, Piper Perabo, George Lopez, Jamie Lee Curtis"
		"Anne Hathaway, Kate Hudson, Candice Bergen")

     (:record-loop :date

      (:day "Mon 26 Jan" "Tue 27 Jan")

      (:record-loop :time

		    (:listing "11:10" "13:30")))))

  "http://www.cineworld.co.uk/cinemas/22")

I wrote the code that reads that defexample macro and generates the data from it in 2 days. When I started I had no idea what the grammar was going to look like and how I was going to accomplish any of the details.

"Last project I worked on had 9154 lines of lisp code out of which 4800 lines are utilities, that I've accumulated and written."

"The utilities cover everything from hash, list, string, files, function, a simple version of prolog, some algorithms like bloomfilter, suffix-tree, a simple bayesian filter implementation and a lot of stuff makes writing code easier."

It is wonderful that you can write all that in 4800 lines in CL, and I know that's not hyperbole or exaggeration.

The problem is the parts covering hashes, lists, strings, files etc. are also being implemented by every other CL programmer, slightly differently. This makes it just that little bit harder to share and understand other people's code, which in turn makes it that little bit harder to build commonly agreed upon libraries for common things, and that little bit harder for people new to the language to get started.

This is all cumulative, and adds up to an actual shortcoming in my opinion. Clojure does a little better job here. For example, by having a common Seq interface shared by lists, arrays, sets, and hashes.

I didn't write a lot of the libraries. Maybe 50% of the code is mine, most the algorithm implementations, the rest of the stuff is picked up from previous projects or existing utilities that lisp programmer's have written.

The reason one uses lisp is because the team is small. Very small like 1 person. So sharing isn't really that much of a concern.

I agree that it makes it harder to start. Which means that very few people stick with it.

Uh, why isn't lisp a functional language? True, it is not pure and side-effect free the way Haskell is, but that hardly disqualifies it.

I mean, OCaml is clearly and widely regarded as a functional language, and it has nearly all the features that Lisp has, including a bundled OO syntax, side-effect IO, and mutable data structures.

In Arc that's

    (= (*hash* "foo") "OH HAI")

And in PLT Scheme it's

  (hash-set! *hash* "foo" "OH HAI")

Seriously? This is a price I'm willing to pay for the symmetry setf buys me. In languges with = for assignment, you are stuck with what the language gives you. In Lisp, you are one "defsetf" away from perfect symmetry between reads and writes.

In addition, usually you are going to hide the hash-table. Instead of accessing it directly, you will probably have an interface like:

    (defvar *customer-cache* (make-hash-table :test equal))

    (defmethod add-customer ((customer customer))
        (cache-customer customer)
        (write-customer-to-database customer))

    (defmethod cache-customer ((customer customer))
        (setf (gethash (name customer) *customer-cache*) customer))

    (defmethod get-customer-by-name ((name string))
        (let ((maybe-customer (gethash name *customer-cache*)))
            (if maybe-customer maybe-customer
                (let ((c (get-customer-from-database-by-name name)))
                    (cache-customer c)
                     c))))

By the time you have more than a one-line example, the length of gethash is irrelevant. You are using your own "API" now:

    (add-customer (make-customer :name "jrockway" :email "jon@jrock.us"))
    (get-customer-by-name "jrockway")
    #<CUSTOMER JROCKWAY {10027D1401}>

The issue is that there is no one solution for all problems and all contexts. ANY given language was written in response to a given set of problems in a given set of contexts. Change one thing to be outside that domain and the fit is less good. Change almost everything and the fit is a misfit.

For example: a 5 lb slege hammer is a very good tool for breaking rocks. Now fix a Rolex Watch with it. Its not so good of a tool, is it?

The answer is understand the problem and its context. Then select the tool that fits good enough (YMMV). Otherwise you will soon find yourself in a world of hurt and still not have a good solution to your problem.

I am sure Lisp has a problem set and a context set for which its a good match. Though I haven't found it yet but then I really haven't looked. The tools I use are good enough for the problems and contexts I have encountered in over 40 years of software development. Thus I have no reason to hammer my head against the rubber wall of Lisp as well as approximately 900 other languages.

Lisp is the best language for generating HTML, XML, writing compilers, or parsers.

PS: Lisp is more a family of languages that share features than it is a specific language.

> , writing compilers, or parsers.

I think the OCaml guys and the Haskell guys(Parsec) might take exception to this.

See: http://flint.cs.yale.edu/cs421/case-for-ml.html http://www.ffconsultancy.com/ocaml/benefits/parsing.html

It's not that CL is bad for writing compilers or parsers (I've written a few parsers using it), its just that blanket statements like "Lisp is the best language for ..." isn't always going to be true.

Some things come down to personal preference, but I don't think OCaml's static type system is more useful than Lisp's approach for parsers / compilers. Granted if your building avionics then type safety is important, but in the more normal case it's just not that useful.

Haskell is a pure functional language which is great for maintaining systems but I don't think it's a real benefit when writing them. If you want to write Lisp as a functional Language you can but when it's useful to break that mold you can.

PS: There is probably a better language out there for this stuff, but I don't know of it.

Anything coming from Jon Harrop is questionable (ffconsultancy.com)

Hince my reference to good enough and rubber wall of Lisp. Lisp is not one thing. Its a collection of collections of things capable of being indefinitely morphed by itself. ANSI C is bad enough without those kinds of problems.

Incidentally I have written several efficient and effective parsers and compilers in both Pascal and ANSI C. I found them quite good enough. I have a hard time seeing Lisp as little more than a big brother to Forth. I see both of them as wonderful solutions to the wrong problems. At least to probelms I don't have nor want to solve.

40 years of software development? I can't help but laugh at your mentioning it as if it really means anything and makes your "arguments" carry more weight.

Reply on news.ycombinator.com