Hacker News Reader

Comment by Touche | Hacker News Reader

I can't figure out what etcd is. Any help?

It can mean different things to different people. I will try to explain how I came to understand how etcd could be valuable for me, which may not be indicative of anyone else's use.

We have lots of configuration automation tools: Chef, Puppet, Ansible, SaltStack, etc. These tools make it easier to orchestrate the modification of configuration files on a node. That is well and good.

However, what is my "source of truth" for how things should be configured? Does the fact that node593958.my.co is my primary database server for app foo belong inside Chef, or another tool, or is it more suited for a "directory" of sorts?

No one can answer that but you. But, if you decide that it shouldn't be embedded in your configuration orchestration tool, you may decide to put it in a key value store like etcd. etcd could expose (via REST), something like:

.../apps/foo/primary_db = node593958.my.co

Configuration management could then key off of that source of truth to configure the application appropriately.

etcd exposes an easy to use REST interface to arbitrary storage of hierarchical keys and values.

Nothing is new under the sun. During the first five minutes I was playing around with etcd, I realized I was effectively reinventing LDAP. This is neither wrong nor right, it's simply a different tool for the job. If you like REST and simple interfaces, it's probably a good tool.

Y'know how you can use ENV on Linux to have programs be configured or tell multiple programs what port something is listening on?

Imagine that, but across multiple servers at once. Plus a few other things that some use cases have been built up for.

"A highly-available key value store for shared configuration and service discovery. etcd is inspired by zookeeper and doozer..." -- https://github.com/coreos/etcd

Except that it's not. Zookeeper was a major pain to get where it is.

It is a distributed configuration management clustering product like zookeeper. It simplifies the creation and sharing of mostly configuration data in clustered or distributed systems.

I don't know what that means or what zookeeper is... Can you explain the problem it solves?

Imagine you're working with a processing system made up of tens or hundreds of computers all (usually) identically configured, networked together and solving small chunks of a large problem. This is software to minimize the effort required to configure all of those individual systems which make up the "compute cluster".

Your web app needs to know what IP address, username and password to connect to the database on.

If it's all one the same box, that's trivial.

But imagine you have several app serves, and several other systems that all needs to talk to the database. Then one day you need to change details of the database server. Now you can either use a tool to update all the configurations (or do it by hand), or you can have all of these apps pull the database config from a distributed directory of some sort.

Etcd is such a directory. Similar to LDAP, but simpler (LDAP requires you to write schemas, and decide which attributes to index on and in set up replication hierarchies). Some people also use DNS for this (you can put arbitrary data in a private DNS server).

Zookeeper is a very specialized distributed filesystem for saving and delivering important details for very large systems. Getting concurrent distributed systems like this right is the equivalent of summited Mount Everest starting by walking from Paris. It was mostly developed at Yahoo and spun off as an Apache project under Hadoop and commercialized by Cloudera. Netflix uses ZK for example.

so you are basically saying that you don't trust the people who designed etcd.

It's trying to solve with marketing what Chef does.

There is more to configuration management that templating files, but in inspection and having other services arriving at other states.

It is not even trying to solve remotely the same issues as Chef. For starters, it is not going to install or uninstall packages or modify files, or nearly any of the things that a typical Chef recipe might do.

Reply on news.ycombinator.com