HNHacker News
TopNewBestAskShowJobs

tobiemh

82 karma · joined May 24, 2021

submissionscomments
tobiemh··on SurrealDB: Distributed document-graph database for the realtime web
Hi johnwoods - Tobie founder of SurrealDB here. We're updating our website so it's clearer, but in the meantime, the best page to see the current functionality is our features page https://surrealdb.com/features

We're also looking to improve our roadmap page soon, which will detail everything we're working on and planning to work on in the near term, and in the future!

tobiemh··on SurrealDB: Distributed document-graph database for the realtime web
Hi user1029384756 - Tobie here, founder of SurrealDB. It's just a team of 2 of us at the moment, with the intention to grow the team very soon!

At the moment (as you have noted) the underlying storage engine is RocksDB (in single mode) or TiKV (in distributed mode), however, although we will always support deployment on top of these solutions, we intend to add our own embedded and distributed key-value stores in due course, which will offer different functionality in the long run!

We're looking to improve our roadmap page really soon to show exactly what we have planned / working on in the near term. In the meantime our features page https://surrealdb.com/features shows details most of the functionality that is already built, or coming in the future.

tobiemh··on SurrealDB: Distributed document-graph database for the realtime web
Hi user1029384756 - Tobie here, founder of SurrealDB. We do plan to support GraphQL integration soon (hopefully before our final 1.0 version). Our features page https://surrealdb.com/features is the best place to see what is built, and what we intend to build in due course! Hope this helps!
tobiemh··on SurrealDB – Document-graph database, for the realtime web
Thanks dodyg. The implementation is already in the codebase. We're just looking to get it built into the Linux version in the next release!
tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi dodyg, it is our intention to provide a fully-managed cloud hosted version of SurrealDB. We'll also offer support to those companies who want/need it. But our primary business model will be the DBaaS platform. We definitely don't plan on going down the same route as RethinkDB did!
tobiemh··on SurrealDB – Document-graph database, for the realtime web
Awesome I'll see you on the RxDB Discord!
tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi yawnxyz - absolutely we intend to have support for Svelte in due course! We aren't Svelte users ourselves, so any suggestions or ideas are very welcome! Do let us know in the community - https://surrealdb.com/community
tobiemh··on SurrealDB – Document-graph database, for the realtime web
Yes you're right. Accessing IndexedDB from WebAssembly needs to go through JavaScript. The only benefit from our perspective is that building it in Rust enables us to use the same logic and code that powers our entire database platform. From a performance aspect, you're right, it still needs to go through JavaScript.

So we don't use any features from IndexedDB at all really. We literally treat IndexedDB as a key-value store interface like RocksDB or something like that. So all of the indexing is done within SurrealDB, and is stored in IndexedDB as a set of binary keys->values (Uint8Array if I remember correctly).

When we want to query a whole table, or a subset of a table, SurrealDB determines what individual keys to load or what key ranges need to be loaded to perform the query.

Would love to chat further about this - it's a really interesting area of discussion for me!

tobiemh··on SurrealDB – Document-graph database, for the realtime web
No not rhetorical - was genuinely interested in any suggestions. Out of interest, if the product was split (like other database products) into a 'core' open-source version, and an 'enterprise' limited license version, would this make any difference?
tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi typingmonkey. So SurrealDB abstracts away the document aspect of IndexedDB so it can be used as a key-value store. We then use that key-value store (in a similar way to RocksDB) to store the data in SurrealDB. You can see our abstraction layer here: https://github.com/surrealdb/indxdb . It's actually pretty performant in our tests so far, and we'll be releasing this soon as a WebAssembly library, enabling SurrealDB to run in the browser locally.

I know rxdb (am a stargazer) so I'm really interested to hear suggestions or thoughts around this area!

tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi danielvaughn thanks so much for the comment! We love CockroachDB and Planetscale! With SurrealDB we intended to build a database where you can use the best of the relational database world, the document database world, and the graph database world. You don't have to choose up front how you store your data, but you can use many different concepts to model and analyse your data once stored.

Then with the ability to connect directly form the client application or from web browsers, you don't have to build any API layer or logic layer in front of the database. Your data access permissions and authentication sit right inside the database (only if you want of course), enabling you to build applications quickly, directly against the database.

tobiemh··on SurrealDB – Document-graph database, for the realtime web
Ok lovasoa, we will definitely give this a thought!!! Just out of interest (and so I can understand your position / usecase better), how would the current license prevent you using it in any open-source projects at the moment?
tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi lovasoa, this is definitely an option, and we're open to suggestions in this area!
tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi AtNightWeCode, we're just a team of 2 developers at the moment, and completely bootstrapped. This is only our initial release, and we have a long journey ahead of us. We realise there is lots of work to do. It would be great to get an understanding of what you mean in more depth :) ! With regards to scalability and high-availability, in distributed mode you run SurrealDB on top of TiKV (and soon FoundationDB), so we don't claim to provide this functionality ourselves (yet).

With regards to the license we wanted SurrealDB to basically be open source, but with the only limitation of not being able to provide a Database as a Service platform. So in a business or enterprise use, there is no limit at all. You can run SurrealDB with as many nodes as you want, and as many users as you want; you can provide a hosted database internally, or to employees, contractors, or subsidiary companies. The only limitation is providing a paid-for, hosted, SurrealDB database platform.

After 4 years, all of our code becomes licensed with Apache 2.0 license.

In addition, all of our libraries, client SDKs, and many of the core components used in the database are completely Apache 2.0 or MIT licensed (https://surrealdb.com/opensource).

tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi revskill, absolutely you could do something like this:

DEFINE EVENT changed ON person WHEN $before.email != $after.email THEN http::post('https://my-webhook.com', $after);

It's very flexible! We're also releasing support for the fetch() function within the JavaScript functions soon, which will enable you to do the same thing from within the embedded JavaScript functions!

tobiemh··on SurrealDB – Document-graph database, for the realtime web
Haha no worries carvking! Thank you!
tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi PeterZaitsev, many of our core components that are used within the database are open source under the MIT / Apache 2.0 license. With regards to the core database code, it is free to use with no limits on number of nodes, data size, number of connections, number of connected users, features, or functionality. The only limit is for providing a paid-for hosted database-as-a-service. In addition, after 4 years, our complete source code for that version release becomes Apache 2.0 licensed. We've tried to keep this as open as possible for developers / businesses / enterprises to use, whilst at the same time protecting ourselves from some of the events that have happened in the database world over the last few years. It's definitely not our intention to just publish our code on Github just to pretend to be open source!
tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi satvikchoudhary! It's a good question. Obviously I know of ArangoDB, but I don't know it well enough to comment on the main differences, so I'll focus on what SurrealDB is aiming to achieve instead.

SurrealDB is aiming to be at the intersection of relational, document, and graph databases, whilst still remaining simple to use with an SQL-like query language, for developers coming from the relational database side. We are only at the beginning of the journey, but SurrealDB is designed to be run embedded, or in the cloud, with the ability to query it directly from a client application or from a web browser (and only access the data that you're allowed to see).

With our native client libraries (coming soon), SurrealDB will be able to be embedded within Node.js, WebAssembly, Python, C, and PHP applications, in addition to running as a server.

We wanted to create a database that people didn't have to manage, so they could focus on building applications, not the infrastructure. We wanted users to be able to use schema-less and schema-full data patterns effortlessly, a database to operate like a relational database (without the JOINs), but with the same functionality as the best document and graph databases. And with security and access permissions to be handled right within the database itself. We wanted users to be able to build modern real-time applications effortlessly - right from Chrome, Edge, or Safari. No more complicated backends.

I'm not sure how all of this compares to ArangoDB, but happy to learn!

tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi carvking! The database can be used completely without our own cloud hosted product (for which there is a waitlist on the site). We're still a little bit away from offering our cloud hosted product, but the product itself can be downloaded, tried, hosted, and hacked on without waiting for our hosted cloud version.

We're definitely not into the 'signup to our waitlist' product, just for marketing purposes. It's only there as some users expressed an interest in registering for it, as they didn't want to run SurrealDB themselves. Just to reiterate, all of our hosted cloud functionality is and will be available in the product which anyone can download and use for free :) !

tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi jpambrun we're definitely open to comments and suggestions around this!
tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi the_duke. The base implementation of that key-value store (written in Golang) can be found here: https://github.com/surrealdb/rixxdb

I don't think the implementation of the ideas in the thesis were added to the public project in the end.

We will be working on this in due course in Rust, however, and the implementation will be completely open source!

tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi gardenfelder, we've got a whole page on our site explaining the thought process and our intentions around the license - https://surrealdb.com/license !
tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi geenat, CockroachDB is an awesome database! SurrealDB and CockroachDB are quite different, in that CockroachDB is effectively a highly-available distributed PostgreSQL database (aiming at that similar functionality), while SurrealDB is aiming to merge the relational / nosql / document / and graph databases together. Hence the functionality is quite different. To make SurrealDB distributed we have built it on top of TiKV (on a side note TiDB which is built by the creators of TiKV is more of a competitor to CockroachDB than SurrealDB is).

We will definitely in due course provide some documentation and comparisons with many of the other databases out there, and also we intend to do benchmarking with the other databases as soon as we have implemented a few of the performance issues that we know about (https://github.com/surrealdb/surrealdb/labels/performance)!

tobiemh··on SurrealDB – Document-graph database, for the realtime web
Lol. Unfortunately only /dev/null is truly web scale :) !
tobiemh··on SurrealDB – Document-graph database, for the realtime web
At the time I implemented the versioning using separate trees on top of the Radix tree. Today I would augment the tree with t he versioned nodes, and would use a Concurrent Adaptive Radix Tree for the base tree. This would allow high concurrency. Then at the base of the tree I would store a temporal link to each tree modification, so that you could access the changes in time order!

Very, very interested in chatting further about this!

tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi endisneigh. We are currently focussed on functionality and stability. There are a number of changes to our code that we know about (https://github.com/surrealdb/surrealdb/labels/performance), that we want to make really soon, and then as soon as those are complete, we'll run some benchmarks against other databases, and detail some comparisons.

It is however hard to perform these direct comparisons, as the functionality is pretty different to a lot of the database out there, but nevertheless it definitely is something we want to get to in due course!

tobiemh··on SurrealDB – Document-graph database, for the realtime web
Thanks so much weddpros! We definitely have some ambitious plans for SurrealDB, and we have lots of exciting stuff to come!
tobiemh··on SurrealDB – Document-graph database, for the realtime web
hi erikpukinskis, obviously SurrealDB can be run on your local development machine, or in your own cloud, so you would never have to connect to our servers for testing. In addition, we are releasing a number of client libraries which (in addition to connecting to a local or remote database server), will also allow you to run SurrealDB embedded i the language of your choice.

Coming soon are Nodejs native module (with embedded database functionality), Python (with embedded database functionality), WebAssembly (so you can run SurrealDB completely in your web browser), and C (with embedded database functionality).

In due course, we want to have native integrations for PHP, Java, and Swift too (and then more to come down the line).

This will mean that you could run tests without having to connect to any external service/platform/server!

Let me know if you have any other questions!

tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi lovasoa thanks for your comment! We did try to stay as open as possible, but unfortunately the license (as you noted) is not an actual OSI approved license. We've got a whole page about why we chose this license, what the limitations are, and how we are contributing to open source (https://surrealdb.com/license).

Effectively though we didn't want to limit the database usage in anyway except for preventing people from offering a paid-for hosted SurrealDB-as-a-Service. Some database companies offer a 'core' open source database, and then put their enterprise features behind an enterprise license (or close source them completely). We wanted to build in the open, but at the same time it is our intention to offer a cloud database version so that developers can get going really easily in due course. This doesn't mean that developers have to use it though :) !

Let me know if you have any other questions or points!

tobiemh··on SurrealDB – Document-graph database, for the realtime web
Hi packetlost this is a really interesting topic to me. I wrote my MSc thesis on this exact topic - https://surrealdb.com/static/whitepaper.pdf ! I've learnt a lot since writing this thesis, and would have many different opinions if I wrote it today, but very interested in this area of research!
Page 1 of 2Next →