aboodman

FYI, Zero uses incremental view maintenance internally as the core of its sync engine:

https://zero.rocicorp.dev/docs/introduction

IVM is what allows the core feature of our DX: that users can just do a fairly large query with sorts and joins, like:

  zero.query.posts.where('authorID', 42)
    .orderBy('created', 'desc')
    .limit(1000)
    .related('comments',
      c => c.orderBy('created', 'desc').limit(10))

... and we sync updates to this query incrementally to the client.

TanStack DB also uses IVM, but only client-side currently.

If you are interested in such things, you can take a poke around the source here: https://github.com/rocicorp/mono/tree/main/packages/zql. Or come find us in Discord: https://discord.rocicorp.dev/

I think this is a fascinating and deep question, that I ponder often.

I don't feel like I know all the answers, but as the creator of Replicache and Zero here is why I feel a pull to the web and not mobile:

- All the normal reasons the web is great – short feedback loop, no gatekeepers, etc. I just prefer to build for the web.

- The web is where desktop/productivity software happens. I want productivity software that is instant. The web has many, many advantages and is the undisputed home of desktop software now, but ever since we went to the web the interaction performance has tanked. The reason is because all software (including desktop) is client/server now and the latency shows up all over the place. I want to fix that, in particular.

- These systems require deep smarts on the client – they are essentially distributed databases, and they need to run that engine client-side. So there is the question of what language to implement in. You would think that C++/Rust -> WASM would be obvious but there are really significant downsides that pull people to doing more and more in the client's native language. So you often feel like you need to choose one of those native languages to start with. JS has the most reach. It's most at home on the desktop web, but it also reaches mobile via RN.

- For the same reason as prev, the complex productivity apps that are often targeted by sync engines are often themselves written using mainly web tech on mobile. Because they are complex client-side systems and they need to pick a single impl language.

> Using Zero is another option, it has many similarities to Electric, while also directly supporting mutations.

The core differentiator of Zero is actually query-driven sync. We apparently need to make this more clear.

You build your app out of queries. You don't have to decide or configure what to sync up front. You can sync as much, or as little as you want, just by deciding which queries to run.

If Zero does not have the data that it needs on the client, queries automatically fall back to the server. Then that data is synced, and available for next query.

This ends up being really useful for:

- Any reasonably sized app. You can't sync all data to client.

- Fast startup. Most apps have publicly visible views that they want to load fast.

- Permissions. Zero doesn't require you to express your permissions in some separate system, you just use queries.

So the experience of using Zero is actually much closer to a reactive db, something like Convex or RethinkDB ().

Except that it uses standard Postgres, and you also get the instant interactions of a sync engine.

At the end of the day people have to care about their job, for a reason bigger than getting a paycheck. Society can coast for awhile when people don't care but things eventually break down.

You can add process but the people running the process have to care. You can add regulation, but then the regulators have to care.

At the end of the day people have to care. And it really has to be everyone, because if one group cares and another doesn't, the one that cares will soon get disillusioned.

Caring alone is not sufficient. You do need process to catch mistakes. But process alone is also not sufficient.

I am confused by this question. Both noms and dolt, and presumably most other prolly tree implementations do what you propose. if they didn't, inserts would be terribly slow.

I was aware of this kind of structure when I coined 'prolly tree'. It's the same thing bup was doing, which I referenced in our design docs:

https://github.com/attic-labs/noms/blob/master/doc/intro.md#...

The reason I thought a new name was warranted is that a prolly tree stores structured data (a sorted set of k/v pairs, like a b-tree), not blob data. And it has the same interface and utility as a b-tree.

Is it a huge difference? No. A pretty minor adaptation of an existing idea. But still different enough to warrant a different name IMO.

The code is also publicly available here:

https://github.com/rocicorp/syncban

Hey HN, I just presented this at React Miami and thought you might be interested. I built an app with three upcoming sync engines, and they run side-by-side and interoperate. Happy to answer any questions.

When you say “rebase” what do you mean exactly? Is it the application replaying the transaction including the interactive code, or is it the database writing the same rows/pages that were written last time?

Yes, this is a central issue in sync. For most applications, sync engines just aren't useful without some solution. Of course you need to validate inputs, support fine-grained permissions, etc., as developers have done with web apps for eons.

In Replicache, we addressed this by making your application server responsible for writes:

https://doc.replicache.dev/concepts/how-it-works

By doing this, your server can implement any validation it wants. It can also interact with external systems, do notifications, etc. Anything you can do with a traditional API.

In our new sync engine, Zero (https://zerosync.dev), we're adding this same ability soon (like this week) under the name custom mutators:

https://bugs.rocicorp.dev/issue/3045

This has been a hard project, but is really critical to use sync engines for anything serious.

No. When we started the project we used prolly trees initially, but we don't need the content addressing feature for Zero and all the hashing was quite expensive. So now we just use a plain immutable b-tree:

https://github.com/rocicorp/mono/tree/main/packages/replicac...

No, Convex is a client/server system like zero, electric, instant, powersync.

If you want a fully decentralized system, check out jazz. It is the best of these currently IMO.

There are so many:

- https://github.com/electric-sql/electric

- https://github.com/powersync-ja

- https://github.com/get-convex

- https://github.com/tinyplex/tinybase

- https://github.com/garden-co/jazz

> Without offline support AND you're doing real time updating of data, then conflict resolution is not a real world practical concern.

I disagree with this. There are many real-world cases where keywise lww does the wrong thing. The article I linked up-thread covers many of them. Even a simple counter does the wrong thing.

This is where robust conflict resolution really matters in these systems, not the long-time offline case people often ask about.

You need robust conflict resolution to make correct software and maintain invariants in the face of write/write systems.

> A backend dev cannot make a general case algorithm that knows that two end users want to keep or throw away in a conflict, because this is completely situational - users could be doing whatever. And if you push the conflict resolution to the end users, then you are asking a lot of those users. They need to be technically inclined and motivated people in order to take the time to understand and resolve the conflict. Like with git users.

I agree completely. In my opinion the ideal offline-first write/write UI has never been built, but the team at Ink & Switch are closest:

https://www.inkandswitch.com/patchwork/notebook/

I think the perfect UX in many cases is that syncs goes ahead and tries to land the offline writes, but the user has a history UI where they can see what happened. Like how many collaborative apps do today.

But importantly in this UI the app would represent branches and merges. But unlike Git's fine grained branch/merge points, in this UI it would literally represent points where people went offline and made changes.

Users could then go back and recover the version of their data from when they were offline, or compare (probably manually in two tabs) the two different versions of the data and recover.

This does still ask users to compare and resolve conflicts in the worst case, but it is not a blocking operation or one that is final. The more common case is the user will go ahead with the merge and sometimes find some corruption. They can always go back and see what went wrong after the fact and fix. This seems like the right tradeoff to me of making the common case (no conflict) easy and automatic but making the uncommon but scary case at least not dangerous.

There also needs to be clear first-class UX telling users that they're going offline and what will happen when they come online.

I'm looking forward to someday working on this, but it's not what our users ask about most often so we're just disabling offline writes for now.

lol wut?

I represented that we ask the dev what to do:

> Zero asks your code what to do

You agree that's what we do:

> You are asking the dev what to do. You are _not_ asking the user what to do.

I get that your actual issue is you don't think that what we do is "the software equivalent of git asking the user what to do". But like, I also said what we do concretely in the same paragraph. It's not like I was trying to hide something. This is a metaphor for how to understand our approach to conflict resolution that works for most developers. Like all metaphors it is not perfect.

FWIW, there is nothing stopping a developer from having this function just save off a forked copy and ask the user what to do. Some developers do this.

Also FWIW, Zero does not allow offline writes specifically because we want to educate people how to properly handle conflicts before we do. I see down-thread this is the majority of your concern.

Zero (zerosync.dev) uses transactional conflict resolution, which is what our prior product Replicache and Reflect both used. It is very similar to what multiplayer games have done for decades.

It is described here:

https://rocicorp.dev/blog/ready-player-two

It works really well and we and our customers have found it to be quite general.

It allows you to run an arbitrary transaction on the sever side to decide what to do in case of conflicts. It is the software equivalent of git asking the user what to do. Zero asks your code what to do.

But it asks it in the form of the question "please run the function named x with these inputs on the current backend db state". Which is a much more ergonomic way to ask it than "please do a 3-way merge between these three states".

Conflict resolution is not the reason why there has not been a general-purpose sync engine. None of our customers have ~ever complained about conflict resolution.

The reason there has not been a general-purpose sync engine is actually on the read side:

  - Previous sync engines really want you to sync all data. This is impractical for most apps.

  - Previous sync engines do not have practical approaches to permissions.

These problems are being solved in next generation of sync engines.

For more on this, I talk about it some here:

https://www.youtube.com/watch?v=rqOUgqsWvbw

Yep - Notion doc answers your question.

Sure happy to give my point of view.

Top-level response: I think Instant is very cool and I have a lot of respect for that team and their work. I've tried to be accurate below, but I know the instant folks frequent HN so maybe they can chime in too and correct anything I've gotten wrong.

1. Instant is a full database, including the backend. Zero is not - it connects to some separate authoritative backend db. Currently only Postgres is supported, but our intent is to support other DBs like MySQL, Mongo, Cockroach, and in the fullness of time even custom distributed systems. Zero's "BYODB" approach has some advantages:

* You don't have to trust us to build a correct backend database (nothing Zero can do can corrupt your data - it's just a fancy cache)

* If you end up not liking Zero you can easily move to anything else that can run on top of PG.

* You can adopt Zero incrementally in an existing project by just moving features over to Zero one-by-one.

* Any code, library, or project that works with PG likely already works with Zero.

* Zero can coexist with other types of PG users. If some non-Zero client changes PG, Zero clients will see it update live. If Zero clients change PG, other clients will see it when they next read PG.

* You don't have to wait for us to build features. Admin UI? Use any of the great PG ones. Data export? It's just PG. Backups? Triggers? FKs? Constraints? Schemas? Migrations? Use all the normal PG stuff.

On the flip side this is a more complicated way to build the system and some of that complexity does leak out into Zero's DX. We're trying to minimize it as best we can, but Zero can probably never be quite as 'sleek' an experience as Instant.

2. Instant uses a datalog/triple style data model and a custom query system. Zero's data model is just relational data, and it uses a SQL-style query system. In practice the API that instant exposes is very relational-like, but I still think that if you know SQL already there's less to learn with Zero.

3. Zero is based on Incremental View Maintenance. When data that affects a Zero query changes, Zero does only a small amount of work to reflect the change. To my knowledge, Instant doesn't do this and re-runs the query (correct me if I'm wrong here Instant folks). This was a specific design choice in Zero because ...

4. Zero was designed primarily to support 'the linear use case': storing dozens of MB of data client-side and keeping it up to date with permissions and partial sync. We did not want to have to run these huge queries over and over.

5. Zero is very focused on just being a sync engine. Instant appears to be aiming to take on more of the suite of features Firebase offers like login, email, blob storage, etc.

6. Instant already has a SaaS live, Zero doesn't.

I hesitate to use the word 'deprecated' because it means different things to different people. We'll continue supporting Replicache but won't add new features to it.

Even if you don't think you'll need partial sync our experience is that almost all projects end up needing either it or permissions. Both of which are difficult with Replicache.

And if you don't need either of those, the the query-based programming model of Zero is just a lot more fun.

I think there are a lot of free hosts that could work?

  - Fly.io has a free tier that has postgres (if you want to debug from your workstation against live pg, which is useful, it's $2/mo for a public IP)
  - Supabase has a free tier (with the caveat that schema changes will resync the replica - which is fine as long as db is smallish)
  - AWS/RDS offers one year free

Those are just off the top of my head. Plus as jitl mentions you can just run the Dockerized pg in any free docker env.

Hello, HN – I'm aaron - CEO and founder of Rocicorp. We build Zero and Replicache. Very happy to answer any questions!

It's even (often) in company's selfish interest to have regulation so that they aren't forced into a race to the bottom.

You should get in the habit of quoting identifiers in pg anyway as there are so many reserved words.

I find it hard to take anyone seriously who makes life harder for themselves for no good reason . Different strokes.

Hello JS was my first language and I use classes because sometimes it seems like the obvious way to model things.

Looking through the source of Replicache, here are some classes we use:

- KVStore

- DAGStore

- Transaction

I mean ... I can of course model these w/o classes, but encapsulating the state and methods together feels right to me. Especially when there is private state that only the methods should manipulate.

We use composition all over the place and rarely use inheritance so I don't think it's just some deficiency of knowledge .

Pre JS classes, the js community emulated classes w/ the prototype chain and that's what I'd have done for these classes if real JS classes weren't available.

You don't have to imagine:

https://www.google.com/search?q=local-first+linear

Makes sense. This is a smart tradeoff.

Also Instant is a database (it has queries and so forth, more appropriate for bigger data). I would put Jazz at the document sync side of the spectrum:

https://x.com/aboodman/status/1843042582672355834

I don't think you're on the right forum if this is your response to somebody launching something new and technically interesting.