Show HN: Meet Dockship
braindump.ghost.io
braindump.ghost.io
If I'm understanding your project correctly, what you've done is create a way to deploy some git-managed code into a Docker container. It's a good idea but I think you are attacking the problem from too high up the stack. The real problem, as I see it, exists one level down: in the Dockerfile and the Docker image. We need deeper control of the environment than a Dockerfile and your app deployer provides. The popular approach is for developers to create their Dockerfile that builds their platform and instantiate a container from this. The problem is that this works great for a small startup with a couple of devs but falls short for larger companies who want to layer configurations on top of one another, applying and enforcing rules across those configurations. Let me explain:
Where I work, we have lots of servers running lots of apps. The apps themselves are managed by various developers but we have standard things that we do across those servers, like install our monitoring agent, applying patches, configuring performance tweaks, etc. The problem with running Docker in this environment is maintaining these things in a centralized manner. Sure, we could create a bunch of Docker images that contain all of our organization standardizations but how do we maintain these? Without Docker, we use Chef and Gangnam-style[1] cookbooks but Chef is not lightweight at all and it's a total pain in the ass for junior guys to learn.
What I think we need is a system of building Docker images that allows dev & ops teams to collaborate and layer different configurations and applications on top of one another in a structured way. It sounds like Chef, but this imaginary application evaluates the "cookbooks" and produces Docker images and Dockerfiles, not bootstrapped servers.
Perhaps Chef will evolve and people will adopt it for deploying Docker. Perhaps someone will write the next-gen Dev/Ops configuration management tool and it will be built around Docker. Perhaps this is already out there and I'm just not aware of it.
[1] http://devopsanywhere.blogspot.com/2012/11/how-to-write-reus...
Docker builds can be nested. That is, you can build a docker image on top of a docker image on top of a docker image, and so on.
So, if you need to maintain a standardized base across several teams in your organization (which is a common requirement):
* 1. Create a source repository for that standardized base with a Dockerfile and any other necessary content.
* 2. Build that base and push it to a registry (public or private)
* 3. Tell your developers to use that standardized base in their own Dockerfiles.
You can repeat that step any number of times.
Does this help?
The image "above" the app is built periodically and the app is built by stitching together the haskell code and the front end js code in a super project with git subtree on every successful test run of ether of those projects. This is all done by Jenkins and each image is tagged by Jenkins build number.
Only problem with this is that the size of the image is getting pretty huge. Anyone have any suggestions on pruning old tags from a docker image and registry?
If you have a lot of Chef, you can get this up and running pretty quickly by using Packer (http://www.packer.io). Packer can build Docker images using Chef to provision them. The next version of Packer (already in git) can tag and push those images to a registry, too, if you want. You probably do, if you want downstream to use Dockerfiles.
Using Packer for the creation of the containers has helped a lot of organizations we work with use Docker while maintaining existing workflows and tools that their employees are trained to use well and that they're happy with. Its a good balance, and easy to replace things with Dockerfiles if you feel the need.
In terms of "allowing dev & ops to layer", the way this looks is you separate what layers you need: clearly you need a base layer (OS), then you want a standard layer (monitoring agents and so on), then maybe you want a stack-specific layer (rails vs node vs whatever), then finally maybe you want devs to be able to put in their application.
Dockerfiles are great and concise for apps (put files here, start by running this command, etc.). Chef/Puppet are great for pretty much the rest of the stack upwards. Packer comes into play there.
It is just an option, at the end of the day, but it can provide a smoother adoption process without having to disrupt too much.
You really don't need packer to use Chef in a docker build. Just install Chef as part of your build, add your cookbooks, and run chef with the arguments of your choice. That's 3 extra lines in your Dockerfile. As a bonus, you get to manage the exact build and version of chef to use, using the exact same syntax and toolbox as for the rest of your software stack. At the end day, chef is just that - software. Why make it a hardcoded special case?
See for example this blog post: http://tech.paulcz.net/2013/09/creating-immutable-servers-wi...
I prefer Chef (or some other provisioners) as a special case because it is a common case. It is just software, but if all your servers (or a majority) are configured using Chef, its nice to have an abstraction to make this easy. The same argument could've been said for Vagrant early on, probably: I could've just supported shell scripts, because shell scripts could install Chef, then run Chef.
Perhaps so, but having the "just set this one to handful of options and you get Chef for free" functionality was a big win for both Vagrant and Chef adoption (as well as other provisioners, I'm just using Chef as an example here).
Likewise, I'm finding that having Docker as a special case in Packer/Vagrant is doing the same: it is educating more people about Docker with a low barrier and bringing more people into an ecosystem I like (VMs, containerization, immutability, or any mix there-in).
I'm super excited about where Docker is headed, and glad containers are finally reaching a point where they're friendlier for laypeople (as in... not super experienced Linux sysadmins) can use them. :)
I've got a branch started to integrate Docker into it. Want to join forces?
Is there anything that ties this specifically to node apps?
I find one thing Docker is very useful for is cases where you have diverse platforms to deploy.