Introducing Cascalog: a Clojure-based query language for Hadoop | Hacker News Reader

Introducing Cascalog: a Clojure-based query language for Hadoop

nathanmarz.com

45 pointsmattrepl9 comments

Amazing how many great looking projects are coming out of the Clojure community. I used to think that Scala was the heir apparent to Java but it seems like Clojure now has more momentum. Clojure has a kinder learning curve than Scala so perhaps that is the reason.

Clojure is definitely a nice language and I really enjoy Rich Hickey's talks. However, I do not anticipate enterprise developers, who are the majority of the Java community, migrating to a language related to Lisp. I agree with your assessment on Scala's learning curve, but its advantage for Java developers is that they can often start out using it like a fancier Java (which is not to say that's what it is).

As for Cascalog: awesome! Nice work, Nathan!

I agree, we have many years ahead of horrendusly conceived Scala code spewing forth from the enterprise.

Scala has basically two learning curves, like most languages that serve well as transitionary languages. To just switch from Java to bad-Scala is easy. To switch from bad-Scala to good-Scala is hard.

Clojure, on the other hand, has no happy path from Java that doesn't require it's journeyman to at least want to think hard about learning something new. There's still a long road from bad-Clojure to good-Clojure (and onto uber-Clojure which includes metaprogramming lispy goodness), but to get to the bad-Clojure state requires substantial effort on its own.

But to get access to tools like Cascalog, boy is it worth it!

I think this is happening because there are so many powerful Java libs that are more useful once wrapped in a lispy/clojure-y wrapper, plus the fact the community always seems willing to work together to help people get past the sticky parts, so the whole can be more productive.

I can't comment on the scala community by comparison, but the environment around Clojure simply seems to make people more productive quickly once they are willing to ask questions and read lots of code to learn. Mind getting to that point can be hard.

I'm tickled that their build system, a slicker replacement for Ant, is called Leiningen.

http://zef.me/2470/building-clojure-projects-with-leiningen

It's a reference to the classic short story "Leiningen versus the Ants", which is a good read:

http://www.classicshorts.com/stories/lvta.html

And the build system looks nice. It has the pleasant feeling I get from scons, with the advantage of being Clojure-specific, so common Clojure tasks are easy.

The info in the link you provided to "Building Clojure Projects with Leiningen" is out of date.

After installing Leiningen, at the command line:

    lein new <project name>

... and that takes care of creating your project directories and the aforementioned 'project.clj' file.

Very neat. It would be even cooler if it had Pig's ILLUSTRATE. I suspect whichever REPL successfully completes an implementation of this: http://infolab.stanford.edu/~olston/publications/sigmod09.pd... will emerge as the leader.

Good idea - Cascading already has a tool to print flows as .dot files so I'll expose this functionality through Cascalog as well.

I'll have to spend some time going through that paper. Being able to generate a small dataset for development would be killer.

Great! The trick will be doing their sample + generate trick. They didn't bother really finishing the implementation for Pig, but when it works its amazing.

Reply on news.ycombinator.com