Hacker News Reader

Comment by SonOfLilit | Hacker News Reader

> Neon allows to instantly branch your Postgres database to support a modern development workflow. You can create a branch for your test environments for every code deployment in your CI/CD pipeline.

> Branches are virtually free and implemented using the "copy on write" technique.

Unless I missed that everyone supports this, this here could be a killer feature and should be advertised higher.

View on HN

Agreed, this direction is underestimated and should be developed better -- we (Postgres.ai) do it for any Postgres with our Database Lab Engine [1], and Neon would bring even more power if it's installed on production

[1] https://github.com/postgres-ai/database-lab-engine

You can get that feature on any Postgres server by installing Citus

Does Citus provide any such storage-level multi-cluster features? I can't seem to find any documentation on that...

I don't understand what you are looking for. Care to explain?

AWS Aurora Postgres supports this to an extent with "clones". You can even clone cross-account. The same copy-on-write stuff applies, so they're relatively cheap and fast. I hope that Google's new AlloyDB will also support it.

https://aws.amazon.com/about-aws/whats-new/2019/07/amazon_au...

There are some annoying restrictions, though. You can only have a single cross-account clone of a particular db per account.

The problem with Aurora's thin clones is extra cost each clone adds

For CI/CD, you want multiple clones running on the same compute power, in a shared environment, to keep the budget constant

It sounds like something you might be able to accomplish with a copy-on-write VFS on top of a Firebird database file. (Not sure about PostgreSQL, but with Firebird, you only deal with one file, so with Firebird, this should definitely work.)

There is an enterprise company called Delphix that does it on top of Zfs - so the idea was in the air.

Instead of duct taping this together with a filesystem we purpose built database storage. The advantage to this is that we can much tighter control execution paths and can profile them end-to-end. Additionally this allows us to integrate with S3 and make it much much cheaper to run.

Technically Firebird just requires a block device, so you might not even need a filesystem.

What are the intended usecases for "branching" a database? Currently, I use separate databases for different environments, are branches better?

Now the most common setup is to copy the production database to the staging once in a while and test migration against staging. With branching, you can test each PR against its own production database branch -- just put branch creation in your CI config. Hence, it has fewer moving parts, is a bit easier to set up, and reduces the lag between prod and staging.

Have a staging / qa env, then fork it for a branch for testing. Much faster than reseeding / restoring.

It's a great feature on heroku for branches, it shares data between review apps. Quite nice.

Reply on news.ycombinator.com