Docker Built-In Orchestration Ready for Production: Docker 1.12 Goes GA | Hacker News Reader

Docker Built-In Orchestration Ready for Production: Docker 1.12 Goes GA

blog.docker.com

48 pointssamber9 comments

Is there any good reason to use this over the much more mature Kubernetes?

Easier to go from dev to prod ?

Not really, though. It's super easy to run a local kube instance (now even cross-platform [1]). In fact, it's much easier at the moment to spin up prod environments for Kubernetes, so I'd argue that Kube still wins in the dev->prod arena.

[1] https://github.com/micro-kube/micro-kube

But you have to learn a bunch of new words (manifests, pods, controllers, etc.) and while spinning up new prod environments is relatively straightforward, maintaining/upgrading one is not. Kubernetes is still a lot more mature though.

I think you mean to link to the now official Minikube (https://github.com/kubernetes/minikube). But yes, this is a big win for getting started and local development.

For me, simplicity beats completeness any day of the week.

With swarm it takes 3 commands to create a swarm of 2 nodes and raise a simple replicated service(not considering installing swarm and ssh'ing into the nodes).

Now lets talk about kubernetes:

With kubernetes I need to edit several configuration files and run 4 commands to start the master, edit more configuration files in each minion run 4 more commands, btw all that without ingress(which swarm has it built in by default, though I hear that it's also coming for kubernetes).

When all that is done I'm finally able to start writing my pods and services(with swarm pods=services so it's less flexible, but easier to wrap your head around).

Writing pods is a challenge on it's own the API changed like there was no tomorrow in the past(maybe that will improve in the future) so most configurations you find on the internet are out of date and will simply not run, nearly every field requires a name tag for no obvious reason, exposing services to the rest of your network seems to require a phd in distributed network systems.

The documentation is often inconsistent with the current version. The customer/community support is non-existent or abrasive. Actually I think this tweet by Kesley Hightower underlines pretty accurately the team attitude towards criticism on the state of the software: https://twitter.com/kelseyhightower/status/73551103792803840...

Finally I must say that all those problems that I have encountered were with creating a kubernetes cluster in our company infrastructure, I have not encountered most of those problems using gcloud. If you use google's infrastructure kubernetes is just harder than swarm, not unbearably obnoxious. Sadly we don't have the option to rent a cloud service for our company.

To sum it up, I think it's great that docker is becoming a competitor to kubernetes their attitude towards quality always impressed me and thus far I'm sold on it, but the best outcome of this is that having docker in the cluster managers competition may help raise the bar for every system.

We've been holding out to leverage 1.12 at my org for simplicity purposes. We only have ~25 bare metal nodes, in a weird middle ground where bootstrapping and managing Kubernetes/DCOS/etc are a bit much but single node docker-compose is too little. The supposed ease to add/remove nodes as swarm workers is very compelling for recovery and maintenance purposes. Basically, complicated enough to solve problems we have and not overly complicated where it creates problems we don't want. Now to give it a try and see if that holds!

Given that Flynn came out with 1.0 the other day, would really appreciate it if someone with knowledge of Flynn and Docker Orchestration / Docker compose describe the differences.

If I'm not mistaken, it's apples and oranges. Docker Orchestration is closer to the Kubernetes/Diego/Mesos level, in that it's assigning opaque workloads to machines in a distributed cluster.

Flynn is more of a PaaS: it has additional logic to wire up your app, inject services, route traffic and so on.

I work for Pivotal, we donate the majority of engineering on Cloud Foundry, so I wind up peeking at lots of other cloud platforms that are emerging in this space. For example, Red Hat dropped their own code and built OpenShift 3 around Kubernetes.

For Cloud Foundry, we don't use any of these HN-famous systems currently. We built and use Warden/Garden because Docker didn't exist at the time. We built and use the Diego orchestrator because Kubernetes, Mesos and co didn't exist at the time. There's a lot of convergent evolution occurring in this area because lots of smart people see the same problems and arrive at similar ideas.

As for scaling up, Nomad currently holds the benchmark porn crown: they posted a 1-million number a few months ago. That said, it depends a lot on what you're counting. We've have Diego running at 10,000 genuinely genuine application instances -- full routing, full logging, full service injection -- in production for some time now. There's work to push the official-we-can-sell-it-to-F1000s-and-not-get-sued limit to 250,000.

Sure, there are are companies who need more than 250,000 copies of their apps running and it may be a bit longer before Cloud Foundry can meet their needs. I mean, I guess they can probably afford to run two copies of Cloud Foundry, if they have to. They might have to hire another operator. Ruinous.

Anyhow. We built an early version of Diego that was to my mind architecturally very elegant but, as I am given to understand, fell prey to a stampeding herd problem as we began to approach heavier workloads and simulations showed it would only get worse unless we went to something simpler.

I'm not sure I agree with the argument in this post that Docker's approach is particularly novel in this area -- separating workers and managers seems pretty standard to me. Diego breaks the work into more than a dozen cooperating subsystems, variously distributed into brains and cells. Diego relies on etcd, which Docker's engineers consider to be an operational overhead. That makes sense because of Docker's engineering assumptions (that someone has only installed Docker), so they kinda sorta have to be autarkic on this matter.

Cloud Foundry -- and by extension Diego -- can rely on BOSH to make operational management of etcd or any other service very close to a non-issue, while pushing that engineering effort out of the core product. I heard recently that a standard OSS Cloud Foundry has 90 (ninety) running processes. I never knew this because I've never noticed, because the upgrades are relatively trivial and almost offensively reliable.

Docker say that read optimisation is a major win from using an inbuilt store, but this presupposes a read-heavy design. A lot of what Diego does is push that work out of the central manager towards the edges. Mesos achieves a similar trick (though it pushes back to the consumer, rather than to the executor). Originally Diego's approach was was more decentralised, with a full auction model (meaning that the central orchestrator didn't need to update any node information at all), but as I said, the stampeding herd of bids on new jobs or processes made it hard to scale. (I am not sure what the alternative was, I'm just an interested spectator.)

Reminder: I work for Pivotal, insofar as Docker is turning back into a PaaS company, we're becoming competitors. I don't work on Diego and I never have, so my understanding of architecture and the course of its evolution is based on snippets of conversation and distant fan-boying. My understanding of Docker's new architecture is from hastily skimming a single blogpost and includes the industry-standard level of heavily and unfairly discounting every other engineer's intelligence and foresight.

So take everything I said under careful consideration.

Reply on news.ycombinator.com