Hacker News Reader

Comment by maxpert | Hacker News Reader

Ideas in CRSQL are remarkably similar to Marmot (https://github.com/maxpert/marmot) except that it doesn't require an extension to be installed, and it can run as sidecar. I can see how having extension makes it simple to intercept writes, and transactions. Since marmot builds on top of NATS, and JetStreams work using Raft, it can easily scale and keep the libsqlite on disk guarantees intact.

View on HN

Marmot looks interesting, but I am curious how it deals with conflicts. What happens if two writes for the same field happen on two different dbs, and they are conflicting?

Since each write is logged into a JetStream. The last writer will win for conflicting changes.

But since you don't have NATS message ordering (from different sources), it's possible that different DBs in the cluster will end up out of sync, right?

So per-node there is an order of changes (read internals doc), and all of changes are sent to JS in that commit order. These changes when replayed will always eventually result in same values.

I am talking about writes that come from 2 different nodes. If 2 different nodes, each making a different write at the exact same time, you would need some global ordering or consensus in order to order them. Otherwise you might end up in a different state on different nodes.

I'd originally done a sidecar route but ultimately went the extension route for portability.

It’s an interesting choice for sure because I heard a lot of ifs and buts for loads of eventually consistent scenarios on why people wanted to not make any changes to their code and don’t really wanna load extensions. Loading extensions is definitely better in terms of transaction control. I did a lot of soul-searching before settling for sidecar approach. In order to not be an interface and stay away from binding complications for various platforms I opted for sidecar. Once done I was already able to scale out of box stuff like PocketBase.

Reply on news.ycombinator.com