Hacker News Reader

Comment by benjaminwootton | Hacker News Reader

I agree that sqlite has a number of similar benefits - openness, table abstractions and concurrent transactions. It’s also a library so close to how delta, iceberg and hudi are implemented.

I’m glad it’s had an uptick in interest recently but I haven’t yet seen it mentioned for analytics yet.

I assume it’s row rather than column oriented?

View on HN

Yes, SQLite is row oriented. It's not a very space efficient format because it also doesn't support compression or compact representations of numbers in binary.

But it doesn't rely on the JVM and a typical JVM ecosystem. It is a big benefit for some use-cases, like dealing with numerical data on the edge.

If you are searching for the analytics equivalent you should check out duckdb.

DuckDB is about querying existing files or am I missing something?

The documentation puts a lot of emphasis on loading/querying existing files, as that's the first thing you will want to do for a OLAP use-case, but in general it's the same as sqlite where you maintain a database file.

Thanks. That's good to know. The file format is not highlighted in the documentation but I found some information about it: https://duckdb.org/internals/storage.html

It seems to be a bit early to rely on it to store data in an object store, but I will do some tests to compare with SQLite:

> The DuckDB internal storage format is currently in flux, and is expected to change with each release until we reach v1.0.0.

DuckDB file format is the compressed column oriented equivalent of SQLite files.

Delta Lake also isn't JVM based. The main implementation is a Rust library with first-party Python bindings.

OLAP or column oriented SQLite is DuckDB.

Reply on news.ycombinator.com