thruflo

ElectricSQL | Senior Frontend Engineer (up-to €100k) & Local-first Software Developer (up-to €50k) | Full-Time | REMOTE in Europe

At https://electric-sql.com we're building an open source platform for local-first software development. This includes doing active-active replication between Postgres and SQLite and working with technologies like Elixir, Typescript and WASM.

We're a remote-first team, distributed around Europe, including the co-inventors of CRDTs. We work a four day week.

More info here: https://electric-sql.com/about/jobs

We (ElectricSQL) have some work underway on a Rust/WASM port of our core client component, that’s designed to open up more language support.

This is a community contributed Dart/Flutter client https://github.com/SkillDevs/electric_dart that may also be a useful reference.

Thanks! We’ve just opened up a call for design partners[0] if you (or any teams out there) might be interested in collaborating with us :)

[0] https://next.electric-sql.com

You can if you acquire a distributed lock in advance, as per https://electric-sql.com/blog/2022/05/03/introducing-rich-cr...

This is a list of research papers on CRDTs, published on the main open source reference site about CRDTs, curated by Martin Kleppmann, Annette Bieniusa and Marc Shapiro.

That’s about as canonical as you can get.

The papers obviously reference the prior work.

What’s interesting to me is to see how much new work continues to build on CRDTs and push the boundaries of what’s possible for AP, causal and mixed consistency systems.

It’s totally valid and acceptable. If equity is part of your comp, you’re investing your time in exchange for equity in the company. The first document any investor would ask for is the cap table.

Beyond the % it also helps you evaluate whether the shares are worth anything. Lots of companies have badly structured cap tables. This impacts investability, which impacts your likely return.

Migrations are a really hard problem for local-first, because (as the article says):

a. clients can be running old code that can’t handle the new schema b. clients can write data locally using an old schema that you want to merge into the new schema

This solution is interesting - note the schema version for all writes and move writes made against an old schema earlier in the order of operations than the new schema migration.

Whether this works is quite dependent on the consistency model and merge semantics. The Cambria [0] approach is to use transformation functions (or “lenses”) to transform the local writes when applied to a backend or node with the new schema.

In either case, you still need an emergency mode that drops writes (loses data) if you can’t handle the transformations or reconcile the merge. For example, if the schemas have moved on too much or you’ve truncated your log so you can’t merge in very old writes.

[0] https://www.inkandswitch.com/cambria/

Yes, exactly.

There are no absolutes but normally fundraising is hard and you need to get 99 ticks out of 100. So starting with `x`s built in makes it tough.

There’s definitely good people in academia and industry who could be great founders but need help cracking the “I can’t start a startup because I have money/life responsibilities” catch 22.

But ultimately if the people who really hustled the company into existence are not the people in the founding team, what confidence do you have that the founding team will raise the next funding round and the one after that without needing their hands held?

And if they have the personality to jump into other people’s vehicles then who’s to say they won’t jump out of this one when the going gets tough?

It’s very hard to align venture building with investability.

The more equity the venture builder takes, the less attractive the business. If the venture builder may need to be washed out, it puts early investors at risk of being washed out with it.

The less the people signing as exec founders on the investment agreement did to actually found the company, the less attractive the team. Evidencing the hustle to start the company is part of what sets a good founder apart.

It’s not so much that 20% of something is worse than 80% of nothing. It’s that the vehicle is compromised from day one.

You can mitigate this with traction but the model is a bit like trying to do a startup with one hand tied behind your back.

You can have transactional causal consistency with an AP system. Combined with CRDTs it’s very powerful.

You also have mixed (“just right”) consistency systems and techniques to introduce and optimise the use of runtime coordination where necessary.

For example see https://electric-sql.com/docs/reference/consistency

Hey, link for the curious: https://electric-sql.com

Hybrid serverless <> local-first is the future. Neon in the client would be so cool.

Erlang is a great language and runtime to build distributed systems in but it doesn’t provide primitives to resolve concurrent overlapping updates.

Hence projects like https://www.antidotedb.eu (CRDT database in Erlang)

From the article, this appears to be an implementation of Making CRDTs Byzantine Fault Tolerant [0] in Rust [1].

[0]: https://martin.kleppmann.com/papers/bft-crdt-papoc22.pdf

[1]: https://github.com/jackyzha0/bft-json-crdt

Yup, open source (https://github.com/electric-sql) + managed replication-as-service. Some of our service code (eg: infra, control plane) is proprietary.

Download size depends on the driver (we support different SQLite drivers for different environments). The web is quite heavy as it loads the SQL.js WASM (as a separate file, it’s not bundled). I want to say ~350-400kb total but I need to check and it depends a bit on how you build/bundle and serve it.

One approach would be to look at the network tab of the browser console when running the web example: https://github.com/electric-sql/examples

Hey, yup, definitely. It should be very similar to the Cordova integration.

You can also follow the generic driver integration instructions [0]. Basically need to implement an adapter interface and compose the various utilities.

Happy to help with this — shout on Discord [1] if you’d be interested in collaborating on it.

[0]: https://electric-sql.com/docs/usage/drivers#generic

[1]: https://discord.gg/B7kHGwDcbj

Yup, it’s a very rigorous system developed over ~10 years with solid testing, benchmarking and formal proofs. However it has some known issues, such as behaviour under high load and not currently handling node failure within a DC.

It’s not production ready and neither are we (we’re in developer preview mode, which is like a public alpha [0]).

There are also other aspects on the Antidote roadmap such as efficiently materialising consistent secondary indexes that are ongoing challenges but aren’t so relevant to how we’re using it as a replication layer.

We are working, alongside others, with the Antidote developers to help fix these issues and generally improve reliability / engineer correct behaviour under load. (Professor Annette Bieniusa, who leads the development, is our Chief Architect). We have a fork at electric-sql/vaxine [1].

We are also taking advantage of some simplifications which mitigate the known issues. We have an #antidote channel in the ElectricSQL discord [2] if you’d be interested in chatting more.

[0]: https://electric-sql.com/docs/overview/faqs

[1]: https://github.com/electric-sql/vaxine

[2]: https://discord.gg/B7kHGwDcbj

Very cool! Would love to swap ideas if you're up for a chat :)

> What I really want to see next is an eventually consistent sync system between browser and server

We’ve made one: https://electric-sql.com

Active-active SQLite to Postgres with transactional causal+ consistency based on CRDTs.

Disclaimer: founder.

Interesting — so basically an overhead until it’s fully native? I was reading the Chrome initiative as a pathway to native. Would that mean it can’t be WASM?

The Google Chrome team is working with the SQLite team to standardise a WASM based SQLite in the browser to replace WebSQL.

https://developer.chrome.com/blog/deprecating-web-sql/

https://twitter.com/chromiumdev/status/1565105522092695553

Which would be pretty cool. Projects like SQL.js and absurd-sql are awesome but naturally a bit rough around the edges and not highly maintained.

I just want to recommend https://overcomingms.org/recovery-program/diet

My wife has relapsing remitting MS and manages it very effectively with this diet.

It’s not easy — cutting out lots of fats - but doing so has a really positive impact: reducing myelin damage, slowing progression and reducing relapses.

Looks great, congrats on shipping :)

Yes, I’ve done this with LiveView and Tauri and it works.

At the time, the barrier to my investigations was LiveView interop with component frameworks. Even Ionic Vue was tricky to get the re-rendering right.

Having a native framework ontop of liveview unlocks shipping it for desktop apps. Bundle a local first database and you have a highly productive stack for building snappy, small bundle, native apps with web technologies.

Ditto is the real deal and the best CRDT database out there atm.

There are expressive CRDT libraries like Yjs and Automerge but these don’t give you persistence.

There are mature databases out there that include CRDTs, like Redis and Postgres EDB, but these have very limited semantics.

Ditto is a proper database with a rich set of CRDTs. It’s well engineered, the team are great and it has a compelling DX / realtime APIs.

Source: working with CRDTs at Vaxine.io

This is really cool. It takes a hard domain — multiplayer — and makes it much more accessible.

I think the docs link is a great intro: https://docs.hathora.dev

I’ve founded companies in the UK, US and EU. I’d recommend the UK.

The cost of a limited company is £12. It’s formed in a day. Use https://www.ukpostbox.com if you need an address.

The legal system is well known and entrepreneur friendly. Accountancy and company admin are simple and relaxed. HMRC is supportive. You can pay dividends on a flexible schedule.

There’s a very large ecosystem of financial support, innovation grants, incubators, accelerators, angel networks and VC firms.

Flip to the US for later stage funding. Nice problem to have.

Needless to say, IANAL, this is not financial advice, crayons, etc.

Also how does the WAL page based replication maintain consistency / handle concurrent updates?

There are a bunch of different practical implementations, as per https://crdt.tech/implementations

https://www.antidotedb.eu is an example of a CRDT database that very much hits disk and preserves the mathematical guarantees. It's based on transactional causal+ consistency, which is formally proven to be the strongest consistency mode for an AP database. Other great examples are https://automerge.org and https://www.macrometa.com

(Disclaimer: building a database system on top of Antidote at https://vaxine.io).

Vaxine | Erlang, Elixir | Europe | REMOTE | Full-time | up-to €120k | significant equity

Hi, we’re looking to hire a Distributed Systems Engineer (Erlang/Elixir, up-to €120k) and a Product Engineer (Elixir, up-to €80k) to work on a rich-CRDT database platform.

Small, highly technical team. Remote with occasional on-site in Europe. Four day week.

More info here: https://vaxine.io/project/jobs

This is really really good. If you can build a collaboration ecosystem around this a kind of GitHub meets Seedlegals then you can create a huge amount of value and disrupt an industry.

Where do I invest?