Comment by gpapilion | Hacker News Reader

Comment by gpapilion | Hacker News Reader

Parent Full thread

I have three fundamental issues with Chef;

1. The node object ends up being two large, which leads to memory issues when a search returns more that 200 nodes (800MB of memory).

2. Chef discourages declarative configuration.

3. Chef lacks a remote trigger mechanism.

Issue one starts to kill you once you have a large number of nodes. Dedicating a quater of the available memory to configuration management seem like a poor financial choice. We've come up with work arounds at my company(generate files centrally, and distribute with chef remote_file syntax), but I still feel they are hacky.

Issue two is more serious. Reindexing on chef only occurs after a node has submitted its node object back to the chef server. This results in incomplete searches until a node successfully completes a run. If you wish to remove a node from a particular role or attribute from a host, you may have a hard time doing so until the next chef run completes.

Problem three is really a result of the expense of running chef. If the memory and CPU costs were lower, there wouldn't be any real issues running Chef more frequently. Some changes I need to go out immediately, some don't matter. I end up back in the world of the SSH loop too often with Chef.

1. We are working on making the search more performance and use less memory.

2. Chef definitely does not discourage declarative configuration. Chef recipes include declarative resource for configuring your infrastructure. Since recipes are an internal ruby dsl, there may be nondeclarative code in them.

3. Chef itself doesn't have a remote trigger mechanism because the. Her run is all about configuring the local node. Nothing prevents you from using the ruby language in a recipe to hook up some kind of remote trigger though. People in the chef community are doing this with projects like Noah and Pylon.

Github.com/lusis/Noah Github.com/fujin/pylon

1. I understand that, and have heard that from you guys several times. The issue is the deserialization from JSON. The monkey patch solution I've see so far stores less data, essentially white listing the attributes for search. It sort of destroys the value of search.

2. I should be more specific. Generally chef relies on the information that ohai provides, not with information enumerated by the administrator. There is a general assumption that the systems are properly configured, and chef is only furthering that, since the hosts provide most of the configuration details. (Yes, you could do stuff with data bags to address this issue.)

3. Thanks for the links. I've not seen those projects previously. I've solved the issue for myself using a much lighter weight solution.

1. Yeah, the stop-gap "solution" is to white list a number of attributes in order to reduce the data set, because in the most common use case we see, there's only a few attributes that people actually care about in the chef-client context. The node's run list, its IP address or FQDN. Really. That is the most common. For like, 90%+ of the use cases out there. Everyone has a unique snowflake and thats cool, but really, not that much.

2. There's no assumption that systems are correctly configured other than they start from a baseline configuration in the most common use case. We have worked with several customers managing existing infrastructures of running systems that had an unknown baseline and Chef was able to automated the pieces they cared about.

Again with "most common use case."

* Why not use transactions? (freebsd jails, linux namespaces) and allow integration testing for the applications in a given environment? Think rspec for integration testing applications and rolling back on failure.

* Why not modularize chef, right now it's a nightmare to get working, emerge makes a graph about 200 nodes long...

I plan on sometime making a puppet / chef alternative because they both seem lacking

You might want to take a look at CFEngine too. It's in that puppet/chef family:

http://verticalsysadmin.com/blog/uncategorized/relative-orig...

And Bcfg2.

Not to discourage you from innovating; just so you know what's out there today.

I've looked at cfengine, bfg and others and they all miss the point of what a configuration management system should do, 1. It should be simple, 2. It should have integration testing., 3. It should be simple. None do this or even try to

Reply on news.ycombinator.com