Hacker News Reader

refseton Insights after 11 years with Datomic [video]

> convincing a query planner to do the right thing requires deep expertise in the query planner

The advantage is that that can be somebody else's job though, and ideally (eventually) an AI's job.

refseton Insights after 11 years with Datomic [video]

> this is partly because most people don't consider 'temporality' as a feature a database should offer

Not sure on your definition of 'people', but I think every business ultimately wants solid auditing and reporting capabilities across their IT systems.

These concerns are only increasing in importance as new regulations demand stronger data provenance, but their implementation shouldn't be reliant on the process of "proper schema design" to get things right first time.

Databases built for the modern world should be making this stuff bulletproof and easy.

(I work on XTDB - and if Postgres already supported temporal tables I possibly wouldn't!)

refseton Insights after 11 years with Datomic [video]

As it happens there's a talk happening next week at PGConf NYC on time travel and system-time versioning in Postgres by the DBOS team: https://postgresql.us/events/pgconfnyc2024/schedule/session/...

But system-time is only half the story. There has been chatter over the years on the PG mailing lists, but there doesn't seem to be much momentum currently towards adding full SQL:2011 bitemporal support to Postgres. Adding support in a way that feels as natural as using Datomic seems unlikely.

refseton Insights after 11 years with Datomic [video]

> that lets you query through time (also called "Bitemporal modeling" with fancier words)

Datomic is actually only 'uni-temporal', it provides a database-wide, immutable "system time" (aka "transaction time") versioning + very effective as-of querying. This naturally falls out of the "Epochal Time Model" (see Deconstructing the Database, 2012). However there is no particular built-in support for any further mutable time dimension, see: https://vvvvalvalval.github.io/posts/2017-07-08-Datomic-this...

System-time is still very powerful though. The way I think of the difference is: system-time versioning is mainly useful for debugging and auditing (and of course for the horizontal read scaling), whereas valid-time versioning is useful for timestamp-based reporting (or other forms of in-application time-travel and modelling 'truth' in business data) where the system-time timestamps are not the timestamps end users are directly interested in, see: https://tidyfirst.substack.com/p/eventual-business-consisten...

refseton Insights after 11 years with Datomic [video]

I suspect there have been a great many examples and attempts at the ORM level - this one with "bi-temporal chaining" springs to mind: https://github.com/goldmansachs/reladomo

And without the ORM layer there's this extension for Postgres: https://github.com/hettie-d/pg_bitemporal

refseton Insights after 11 years with Datomic [video]

The original RDBMS vision was very explicitly for the users (both developers and analysts) to not have to be experts in their own database in order to achieve useful work, and without needing to think about procedural/3GL code from the get-go. In the intervening years query optimization has gotten a lot better, and hardware shifts have only worked in favour of this vision, but there's still a lot of work to be done before databases are truly "self-driving": https://www.cs.cmu.edu/~pavlo/blog/2018/04/what-is-a-self-dr...

Until that's the case I can understand why people are tempted to bypass this traditional RDBMS wisdom, especially if they have a very strong conception about their data models, access patterns, and need for scale (e.g. see also Red Planet Labs 'Rama').

refseton Insights after 11 years with Datomic [video]

> If there were a Datomic for Elixir, I'd probably use it.

XTDB may possibly of interest in that case - there's been some recent chatter about adding support for Ecto via Postgres compatibility: https://discuss.xtdb.com/t/elixir-ecto-thread-from-slack/490

(I work on XTDB)

refseton Insights after 11 years with Datomic [video]

> In databases like XTDB, the order of :where clauses doesn't matter since the engine is able to optimize them

Rich's observation was that query optimizers often get things wrong in ways that are hard to predict or control, but he's not fundamentally opposed to their use. That said, building a decent optimizer is a huge undertaking and I think they took the right decision to not attempt to bundle that sort of complexity into the original vision for Datomic otherwise they might never have shipped.

The state-of-the-art commercial engine for query optimization and execution algorithms in this 'triple' space is probably https://relational.ai/

refseton Reactive Relational Algebra

Also in Clojure, and reactive, and relational: https://github.com/wotbrew/relic

> Functional relational programming for Clojure(Script).

refseton PostgreSQL 12.3 Jepsen Analysis (2020)

I was surprised that this Jepsen report has never been discussed here before, but it really captures just how problematic the history of 'safe' database isolation levels has been. Some excerpts:

> For the last nine years, PostgreSQL’s “serializable” mode has justifiably claimed to offer serializability.

> [but] PostgreSQL’s “serializable” isolation level isn’t serializable: it allows G2-item during normal operation

> [...] These cycles are precisely what PostgreSQL’s SSI implementation is meant to prevent!

> This code has gone essentially untouched since the introduction of serializable snapshot isolation in 2011.

Making things appear to be serializable is really hard.

refseton Datomic and Content Addressable Techniques

InstantDB (YC S22) launched a few weeks ago https://news.ycombinator.com/item?id=41322281

refseton Haskell Postgres Stored Procedures

In case you weren't aware of TutorialD / Project:M36 already: https://github.com/agentm/project-m36

> Unlike most database management systems (DBMS), Project:M36 is opinionated software which adheres strictly to the mathematics of the relational algebra. The purpose of this adherence is to prove that software which implements mathematically-sound design principles reaps benefits in the form of code clarity, consistency, performance, and future-proofing.

> Project:M36 can be used as an in-process or remote DBMS.

> Project:M36 is written entirely in the Haskell programming language.

refseton K1 Buys MariaDB

Native bitemporal tables: https://mariadb.com/resources/blog/temporal-tables-part-5/

refseton Show HN: Visual DB – Web front end for your database

Thanks, I've not used Airtable enough to appreciate the comparison entirely but the focus on form building makes a lot of sense. Google Forms sets a low bar and the next best choice isn't obvious to me. Congrats on the launch!

refseton Show HN: Visual DB – Web front end for your database

How would you describe the differences with https://nocodb.com/ ?

refseton Sqlc: Compile SQL to type-safe code

Possibly close enough: https://github.com/omnigres/omnigres

refseton Sqlc: Compile SQL to type-safe code

Building on top of IntelliJ's parser and 'PSI' stack as a headless engine sounds slightly mad at first but seems to provide a lot of leverage, as demonstrated by the number of dialects the team is able to support via mixins. By contrast sqlc appears to require a lot of code to support each additional dialect.

refseton Sqlc: Compile SQL to type-safe code

Yep sqlc is more akin to Kotlin's SQLDelight https://github.com/cashapp/sqldelight

refseton Rɐbbit Dynamic Datascapes

It's not terribly obvious while skimming the page but there are some videos linked via a few of those images ("Basic SQL ▶" etc.) and the author has also uploaded loads of narrated demos over the recent past, prior to releasing the project: https://x.com/ryrobes

If I were to attempt a summary myself... RVBBIT is a SQL-powered relational programming canvas with a visual Clojure REPL and full-stack UI framework.

refseton Core: an experimental new way to write videogames

Reminds me a little of this Flappy Bird demo which makes use of DataScript's Datalog rules https://frankiesardo.github.io/minikusari/#!/minikusari.tuto...

refseton Show HN: InstantDB – A Modern Firebase

> What were the show stoppers in adopting solution like XTDB?

#1 would be the lack of a production ready DBaaS (like Aurora offers for Postgres), if I had to guess.

refseton Zettlr: Note-Taking and Publishing with Markdown

I agree, fwiw. It is better to introduce new words than to risk being so provocative. Still though I think Zettlr is founded on sensible ideals.

refseton Zettlr: Note-Taking and Publishing with Markdown

I actually don't think the headline captures the point succinctly, because the definition of 'political' being used is obscure and quite specific:

> "Political activity is whatever shifts a body from the place assigned to it or changes a place's destination." (Rancière 1999, S. 30)

> What Rancière means by that is that politics occurs whenever a thing is not in its assigned place. This is perfectly valid for software: As soon as a person diverts from what is expected from them, software aborts with an error or "corrects" the "wrong" statement automatically. So at least from the view of Jacques Rancière, software is utterly political. It has a significant impact on our daily lives and we cannot do anything against this. And this is a problem that especially corporate software shows.

refseton Zettlr: Note-Taking and Publishing with Markdown

The motivations behind the project are quite interesting and well argued: "Developing Open Source is a Political Act"

https://www.zettlr.com/post/release-developing-open-source-p...

refseton Electric Clojure v3: Differential Dataflow for UI [video]

In the video Dustin also makes the point that in v3 all the client and server annotations can now be kept fully isolated from the parts of the code that model the essential complexity (i.e. any dynamic scoping or function parameters used by a 'pure' function are also network transparent), and this dramatically reduces the amount of global coupling across the codebase.

refseton Making database systems usable

The 4GL dream hinges on figuring out how to transition from #2 -> #1 seamlessly. The spectrum is wider and more complex than "Excel -> Postgres" but even solving that journey would be a good start. It could save us all from a whole a bunch of ~needless engineering work.

> it seems very possible to go from #1 -> #2 mechanically, and hand that off to the users who want to play with it

NocoDB seems like a reasonable attempt of taking this approach for Postgres.

refseton Just use Postgres

If Postgres already had decent temporal table support (per SQL:2011 system time + application time "bitemporal" versioning) we never would have gone down the road of building XTDB. From the perspective of anyone building applications on top of SQL with complex reporting requirements in heavily regulated sectors (FS, Insurance, Healthcare etc.), "just use temporal tables" would be the ideal default choice. To get an idea of why, see https://docs.xtdb.com/tutorials/financial-usecase/time-in-fi...

refseton Postgres.new: In-browser Postgres with an AI interface

There's a video overview here: https://www.youtube.com/watch?v=ooWaPVvljlU

Really impressive stuff - congratulations!

refseton Amazon's exabyte-scale migration from Apache Spark to Ray on EC2

Is there anybody trying to build a SQL database on Ray yet? Asking for a friend.

refseton Amazon's exabyte-scale migration from Apache Spark to Ray on EC2

Yep backtesting is critical step in the deployment of new trading algorithms.

Related, I rather enjoyed reading this other thread from June: "Ask HN: Is KDB a sane choice for a datalake in 2024?" https://news.ycombinator.com/item?id=40625800