Hacker News Reader

Comment by chubot | Hacker News Reader

I watched the first part of this, and this is apparently the inventor of Scala talking about the Scala compiler.

He was talking about how deep the transformation pipeline in scalac is, compared to javac.

I would say that something close to the opposite statement is justified: databases are interpreters, e.g. sqlite is certainly an interpreter with a storage layer.

I didn't watch enough of it to see why he thinks that compilers are databases. Where's the storage layer? I'm interested in compilers and databases, but TBH I am not that interested in Scala, so I stopped watching.

View on HN

The storage isn't the important part. I think his point was more about _functional_ databases than your typical RDBMS in the sense that the experimental Scala compiler needed to model time (in terms of compiler passes and phases) in order to be able to answer "What is the value/type of this thing _at a given time_?". TBH, the talk was very little about Scala and mostly about compiler architecture.

What's an example of a "functional database"? Datomic? What's the most deployed functional database?

It's misleading to say that compilers are databases if they're not anything like databases people use and know. Probably a more accurate title would have been "Compilers are like functional databases".

Storage is obviously an important part of database architecture. If a compiler doesn't deal with storage, there's only so much resemblance that a compiler and a database can have.

I think "functional database" can be seen as a wider data pattern than just a datastore that exclusively implements that pattern. It's a design principle where a piece of data once created is never changed. That leads to event sourcing or Twitters "lambda architecture".

The compiler is a database in the sense that it has a memory across incremental invocations. So the 'storage' happens to be in memory but could be on disk.

I'm definitely not a DB guy, but Datomic was the first thing that came to mind. You could also include databases that require linear consistency like etcd if you consider "functional databases" to be any database that represents state as a function of time.

There was an early slide that explicitly said FRP and functional databases were inspiration for the compiler design, but I do agree that the title should have reflected that fact.

He means that compilers create an in-memory database, representing your code, which could be put in an actual database.

Not that compilers can be used as a general DBMS. I think.

Thanks for the TL;DR! I see the point, it's not that new at all. For example, Coverity tools are using sqlite to store the huge pile of IR.

Hm, I always thought that the sql-layer is a communication layer on top of the actual db. So sqlite as a database doesn't interpret much. On the other hand the sqlite shell tool of course is an interpreter for SQL.

I would say both statements are justified. Transitive property being what it is

I think you mean "symmetric" rather than "transitive", and the "is a" relation isn't symmetric. (We know this from OOP -- i.e. the Liskov substitution principle.)

A database is an interpreter because it has a lexer, parser, byte code compiler, optimizer, interpreter loop, etc. It also has a storage layer (i.e. the B-tree implementation, paging, etc.).

A compiler (like javac) or interpreter isn't a database because it doesn't have a storage layer.

Even a "functional database" has to have a storage layer!

Once you add IDE and REPL support (Which Scala does) you need a storage layer as an optimization. Odersky spends most of the lecture time detailing resulting performance goals and effects on internals to support these features. This leads him towards a normal-form topology to minimize data duplication, of course, which is why he gave it the title "Compilers are Databases".

Reply on news.ycombinator.com