17 karma · joined September 9, 2021
Oh but they can't. If you tried it, then you surely know that both OT and CRDTs need to consider the entire change history at some key points in order to derive the current value. Diff sync doesn't suffer from the same issue, however the way it keeps track of client shadows introduces writes on the read path, making it horribly expansive to run at scale.
Are you seriously trying to say that the database you created in a scripting language that uses linear scanning of arrays is 'unmatched' compared to high performance C++?
It's not about the language, but about the underlying algorithm. Yes, JS is slower, and surely linear scan is slower than typical DB queries. But what GoatDB does, which is quite unique today, is it's able to resume query execution from the last point a query ran, so you get super efficient incremental updates which are very useful when running on the client side (clients tend to issue the same queries over and over again).
- The queue introduces delays so this doesn't play nice with modern collaborative editing experience (think google docs, slack, etc)
- Let's say change A set a field to 1, and change B set the same field to 2. GoatDB allows you to easily get either 1, 2 or 3 (sum) or apply a custom resolution rule
Your only choices before goat to solve this were: Operational Transformation, raw CRDTs or differential synchronization. GoatDB combines CRDTs with commit graphs so it can do stuff other approaches don't at an unmatched speed
The problem really is about concurrency control - a DB creates a single source of truth so it can be either on or off. But with GoatDB we have multiple sources of truth which are equally valid, and a way to merge their states after the fact.
Think about what Git does for code - if GitHub somehow lost all their data, every dev in the world still has a valid copy and can safely restore GitHub's state. GoatDB does the same but for your app's data rather than source code
- Back when we started, only Deno had the ability to compile to a single exec
- We're using deno's module resolution which is superior in every way (with an ESBuild plugin)
- Deno's filesystem API
All of the above can be implemented for other runtimes, and it's definitely on our roadmap
Pros: - Branch based deployment so multiple versions can coexist nicely in prod - Completely synchronous API that fully hides the networking - Clients can securely restore a crashed server - Your DB functions as a signed audit log that prevents cheating (similar to blockchain) - Stronger consistency guarantees that simplify development
This is an experiment in flipping the traditional design on its head and pushing most of the computation to the client using an architecture similar to what Git is using, but for your actual application data.
You get all kinds of nice byproducts from this design like realtime collaboration, secure data auditing, multiple application versions coexisting on different branches in production, etc etc. It's really about pushing the boundaries of what's possible with modern client hardware
GoatDB has backends for both OPFS and IndexedDB