Interview at the Clojure Conj 2011 with the creator of Noir/Korma/Pinot | Hacker News Reader

Pinot

mostlylazy.com

44 pointscemerick9 comments

I haven't used Korma yet, but I spend an hour or two a day using the Noir framework: hits the sweet spot being low level for fine control but is still very quick/agile/convenient. I am releasing a web app for a new business hopefully in February that uses Noir. I am also working on a Noir based web app for a customer.

Now that I have gone through a small learning curve for using Noir, Stripe, and getting a little better writing Javascript I am also going to convert a half complete Java/GWT project to Clojure and Noir.

Should you have any questions, I'll be lurking :)

Lurk again tomorrow, after my commute. :)

Appreciated the link to a clojure based podcast in general though, and noir is far too long on my list of things to check out.

I haven't had a chance to listen to the podcast yet, but I do have a question and topic that I'd like to bring up again.

Korma vs ClojureQL. In previous online discussions you've stated that you feel that ClojureQL is "not the right abstraction." Could you expand on this a bit? This seems like an odd statement to make considering that relational databases are based on relational calculus as is ClojureQL. In a sense, ClojureQL is not much of an abstraction at all.

This brings up a second but related topic: performance of the generated queries. I understand that one of the biggest complaints about ClojureQL is that it generates rather inefficient queries. Both ClojureQL and Korma fit into a rather strange place of compiling a clojure DSL down to SQL, which itself is compiled by the database. This brings up a lot of interesting theoretical questions that I haven't really seen addressed.

Given that ClojureQL is based on relational calculas, it's interesting that there needs to be a compilation phase at all. Is there not a direct mapping between CQL and SQL? If not, is CQL actually more expressive than SQL? How does Korma differ? Korma must compile down to SQL, and thus doesn't avoid the relational calculas abstraction at all. The syntactic sugur for foreign keys (relations such as has-many) still need to be compiled down to basic SQL and it would seem that there might be more query efficiency issues or even database design issues involved in this sort of abstraction than in ClojureQL.

In short, it's not at all clear to me what Korma provides that is better than ClojureQL, why ClojureQL has query efficiency issues, and especially your comments about the relational calculus abstraction.

It depends on which direction you approach the problem from. Are people approaching their database from the stand point of pure relational algebra? Or are they approaching it from the stand point of "this is how SQL allows me to interact with my data"? In my experience, it's very definitely the latter. As such, providing an interface that attempts to model relational algebra explicitly and treats SQL as an implementation detail doesn't cleanly map to how people actually think about the problem. This makes the abstraction obtuse and significantly more complex. At the simplest possible level, when I go to look up "how do I do x in postgres," the answer will be in SQL, not relational algebra. Why force users to remap the way they think about the problem? Moreover, what do we actually gain by trying to map relational algebra to SQL? Why not work in SQL directly? That's what Korma does.

CQL generates lots of subselects in places that they aren't needed. I'm not looking at it from the stand point of the time it takes to generate a query, but instead how long it takes the generated queries to run. Subselects are slow. Generating queries that use lots of them is going to be slower than a system that doesn't. Could CQL better compile down to SQL? Probably, but it doesn't currently.

In terms of expressiveness, both allow for composition of SQL components over time, which definitely beats SQL itself. I would argue that Korma has an edge over CQL here simply because it allows you to directly craft the queries to operate exactly the way you want.

I should also point out that there's no reason these two can't exist together. When you really think about it, Korma is like ring for SQL and enables a host of higher level abstractions to be built on top of it. To that end, it would actually be fairly easy to build CQL on top of Korma :)

The issue I have here is that I think at their core, both solve the same problem in nearly the same way. The use of SQL-like syntax versus CQLs more mathematical terms like project is a valid point, but ultimately doesn't at all change the abstraction. SQL itself is largely just a different syntax for relational calculus.

I suppose I'm seeing another Lein vs. Cake situation here and feel that it would be nice if efforts were concentrated.

Both CQL and Korma need to make some changes to make fragments composable. The only really interesting difference is the issue of subselects and why CQL generates a lot of them in it's translation to SQL. From a theoretical standpoint, subselects can often be rewritten as joins and many databases do this as part of their query compilation process. It doesn't seem like an unfixable problem.

For example:

  (-> (select* "user")
    (fields :id :username)
    (where {:email [like "*@gmail.com"]})))

This can be written in CQL (I think):

  (def user (table db :user))

  (-> (select user)
    (project [:id :username])
    (where (like :email "*@gmail.com")))

So the difference is only renaming project to fields? Niether keywords are in SQL. Other syntax differences like limit instead of take are similarly minor. It almost seems like you could just def some aliases and the two libraries would read nearly the same.

It really feels like a part of Korma and ClojureQL should be the same library with the entities and has-one type relations built on top of it. Choice of keywords shouldn't be a massive stumbling block to collaboration, although the subselect problem in CQL should certainly be addressed. I do hear what you are saying, but I can't actually see it in the implementation.

Edit: I don't mean to be overly critical. :) It's just that the two libraries seem so very similar to me!

If you look at their implementations, they're very different :)

But as I said, they could converge - CQL fits nicely as a higher-level interface over Korma. And for what it's worth, I didn't build Korma for no reason... an awful lot of people have wanted an alternative to CQL. If for no other reason than to have something that is being really actively maintained.

That much is clear, and I think the differences are probably the interesting part. :) I'll try to set aside some time to look at the implementation details of the two libraries.

My favorite blog post (not mine) discussing this is:

http://thinkrevoactevo.blogspot.com/2011/11/sql-in-clojure.h...

but it also does not address the fact that relational algebra and SQL selects are very closely related things. I think that there is a very nice theoretical issue hiding here realting to composable sql fragments. If I come to any interesting conclusions I'll message you. Part of my day job is writing a new type of query compiler, so this is an interesting topic to me.

Reply on news.ycombinator.com