Hacker News Reader

Comment by akie | Hacker News Reader

I made this, and have been using it in production and for local development for the past year or two. It's been immensely helpful to quickly save or easily restore the database to a previously know 'good point'. I have also been using it to quickly load staging databases into development and vice versa. So far it's MySQL only, but the connection points for other databases such as PostgreSQL are in place.

It's main goal is to simplify development - it's not intended as a replacement for writing database migrations or to work with production databases. I would be grateful for any feedback, preferably in the form of pull requests ;-)

View on HN

I like this pattern of being able to restore snapshots in your development environment. For the uninitiated, docker actually has all these functions. For example, if you create a container for postgres and mount a volume, you can create a snapshot of the volume for distribution for your team so they can mount/unmount anytime.

I was under the impression container file systems make running a DB in a container a bad idea? Or is it just in performance critical situations?

Doing so into the main container filesystem (which is usually an overlay/union stack) is probably bad for performance.

But a bind mount (which is usually the recommended way to do it so that data survives the container lifespan) will avoid that and just leave you with whatever "problems" the host filesystem has re: database files.

> you can create a snapshot of the volume

how? note that docker commit/export does not include volumes

This functionality for Docker was proposed by NetApp (https://netapp.io/2017/06/22/snapshots-clones-docker-volume-...)

There was also a github issue with numerous people voting for it (https://github.com/moby/moby/issues/33782)

But for whatever reason, the docker team apparently decided not to build it. We still don't have a properly easy way to snapshot or share docker volumes.

So I came up with my own method.

Basically, you have to mount the volume and then tar the parts of it that you want snapshotted (typically your data directories). You can do this with any linux machine image. Here's an example (where "foo" is the docker container you want to snapshot):

  mkdir -p docker-volume-snapshots && docker run --rm --volumes-from foo -v $(shell pwd)/docker-volume-snapshots:/docker-volume-snapshots debian:stretch-slim tar -czvf /docker-volume-snapshots/snapshot.tar.gz /var/lib/my_data_folder

Then, to restore the snapshot, here's what you do:

  docker run --rm --volumes-from mysql8.0 -v $(shell pwd)/docker-volume-snapshots:/docker-volume-snapshots:delegated debian:stretch-slim /bin/sh -c "cd / && tar -xvf /docker-volume-snapshots/snapshot.tar.gz"

(Edit: Don't forget to start up/initialize and then pause/stop the running "foo" docker container before you run these commands to snapshot the volume)

Looks interesting but I doubt that'll be a consistent snapshit, right? I guess you could combine it with `docker pause`

oh, yeah, I forgot to mention that I stop the running docker container before I do this

FYI, in the Titan project (https://titan-data.io), we solve this problem by running a privileges storage container (with ZFS on Linux) that then is able to export volumes into other containers, giving us ZFS snapshots and clones under the hood. This was inspired by the approach taken by dotmesh (https://dotmesh.com/) but is definitely tailor-made to our use case. If someone was sufficiently motivated, I'm sure you could build a different CLI on top of our server and use it simply to manage docker volumes without the Titan trimmings (e.g. push and pull).

Interesting utility. I can see a few really exciting use cases for this.

If I had one wish it'd be native support for syncing generated snapshots to S3 or Google Storage for easy transfers between authenticated systems and users. Maybe I'll take a look at building out a PR for that over the weekend if that's okay

Yes, by all means please do. Simplicity of use is very important to me, so make sure it's generally applicable and easily understandable, but other than that - go for it! Awesome.

Very interesting, my go-to workflow so far has been creating backup copies in phpMyAdmin but this seems much more convenient and reliable. I'll give it a try soon!

Reply on news.ycombinator.com