Mozilla Circus 0.10 released | Hacker News Reader

Mozilla Circus 0.10 released

blog.mozilla.org

86 pointsrbanffy30 comments

All the web console screenshots are missing

http://circus.readthedocs.org/en/latest/for-ops/circusweb/#c...

Yes thanks - I have fixed this and triggered a build - it should appear in a little bit

Still not seeing 'em here.

yeah the build is staled on readthedocs ... trying to see how to fix that. They will eventually show up sorry. You can find them in the docs/ directory in the repo

Total noob question here. I am not from an ops background...

To paraphrase the docs, it's possible to manage a process and specify how many of that process I want to spawn (i.e. numprocesses = 5). I can also manage sockets and pass the managed sockets' fd to the managed processes.

First question, what is a typical use case for wanting multiple instances of a specific process?

Second question, if I am running multiple managed processes, how can they all bind to the same managed socket?

I guess there are some fundamental gaps in my understanding.

> First question, what is a typical use case for wanting multiple instances of a specific process?

distribute the load. For example I can run several "redis workers" that gets jobs to do in a redis queue.

> Second question, if I am running multiple managed processes, how can they all bind to the same managed socket?

if you bind a socket and then forks 10 child processes, they can all accept connections on that socket and let the operating system do the load-balancing.

This is exactly how the Apache pre-fork model works.

Awesome, thanks for answering my questions.

Think of circus as a more modern and intelligent version of xinetd. It binds to the socket and then hands off requests to the workers. Here is why this is awesome...

So we use circus to maintain about a dozen apps and a few of them are django apps running under Mozilla's chaussette wsgi server. When we do an autodeploy via jenkins, jenkins will run circusctl to stop that worker, replace the code, futz a symlink, and then start the worker again. We front the apps with apache 2.4 (because it is as fast as nginx) + mod_proxy. That gives us ldap access control through apache and shiney LTSV formatted logs that tools like logstash or splunk can easily consume. We then front apache with a virtual ip provided by a tool such as keepalived for moar web scales.

For all in-flight requests, they will just hang as circus proper will hold onto the connection keeping the socket open. It will pass the socket to the first worker started once it is restarted. So during a deploy, a request might hang for a few milliseconds, but it never blows up. It is awesome in that it allows you 0 downtime deploys (at least for our django web stack it does).

Circus is a super elegant piece of software. Kudos to Mozilla for finally making a release. I'd been bugging them for awhile: https://github.com/mozilla-services/circus/issues/561

Edit:

Example of a 0 downtime restart config using uwsgi (instead of chaussette): https://github.com/scottkmaxwell/circus/commit/ef0000fe4ef3c...

We've been using circus in production for nearly 6 months, and I've been very impressed. Configuration makes sense, and it's been rock solid for us. If you've ever found yourself frustrated by supervisord, or upstart, I'd recommend it.

What is the major difference between circus and tools like supervisor?

Author of Circus here.

Major differences:

* Circus uses ZeroMQ, Supervisor uses XML-RPC. * Circus is Python 3 compatible, Supervisor is Python 2 only * Circus will let you manage sockets - see http://circus.readthedocs.org/en/latest/for-ops/sockets/

Thanks for the clarifications. Just for the record it seems that Supervisor will roll out Python 3 support soon https://github.com/Supervisor/supervisor/issues/110. It's good to have it here and now though.

That's great to see Supervisor is moving forward on the Python 3 support.

Scott is actually the person who ported Circus to Python 3, so double kudos for him.

When I last compared them, circus looked more specific to managing something like wsgi workers while supervisord looked a little more "general-purpose".

This was probably a wrong or overly simplified conclusion, but I know supervisor and it hasn't let me down yet - so for now I see no compelling reason to switch. Maybe it's one of those "you just have to try it" moments? :)

Supervisor is solid and mature, that's for sure. If not for the sockets feature, I would say that you would want to move to Circus if:

- you need python 3 - you have a complex start/stop sequence where you need to add some custom code - you manage many servers and want to quickly jump from one to another in the same web dashboard / CLI.

Otherwise, they both provide roughly the same things I'd say

I watched a 60 minute talk from Tarek Ziadé a few months ago in EuroPython 2013 which was very interesting. His point regarding the missing features that made them launch project Circus were realtime stdout/stderr, realtime stats, powerful web console, remote, clustering and scalable.

You can check out the slides here: http://blog.ziade.org/slides/pyconireland2012/

What are the benefits of using something like this if your OS comes with a modern init like systemd?

Mostly portability: you can provide a single configuration to manage your stack no matter what the system is

It's also easier to run in the user space if you are not root

I use both! systemd starts and ensures circusd is running at all times. Then I can have circus running as an unprivileged user and have jenkins not need to be root to manage deployments. Look up a bit for my other comment describing (from a high level) how this is setup.

I really appreciate the emphasis on collaboration, especially with new contributors. I might be replacing some of my cobbled-together monit setups.

So you password protect the HTTP interface; is there anything that protects the 0mq interface?

I used to for a PaaS and we made a mistake allowing one component to run any command what so ever. This can seem appealing; but it's a security risk extraordinaire.

Security keeps taking a back seat to useability :(

zeromq is not secured - but they are working on adding security in the protocol.

In the meantime the safest way to avoid any security issue is to run IPC-only zeromq sockets or to set properly a firefwall and to use an SSH tunnel. There's such an option in Circus, where you can pass the ssh server to circusctl.

also, read up http://circus.readthedocs.org/en/latest/design/security

just noticing the comment 'When a server is being stopped, we need to make sure we don’t lose any incoming data.'... the potential for data lose is already present with the use of redis server ?

I guess I am missing something.

I don't know all the details on the project and if there are some single point of failures, but as far as Circus is concerned, a properly working graceful shutdown is a must-have when your app is getting data to process.

e.g. 1/ notify the world you don't accept data anymore 2/ process what you have enqueued 3/ shutdown. In Fabien's use case 1 -> 3 can last for over 5 minutes.

thx, I think I understand what you are saying,

my (perhaps snarky) comment was more to do with 'why be so careful' with the potential data failure (that circus handily addresses) when data is being placed in a 'leaky bucket' (redis) in the first place.

Circus looks great, although there doesn't appear to be an email alert plugin (yet?). I use this with Supervisor to alert me of restarts/crashes.

The web interface is certainly pretty, though.

Yeah we did not ship a plugin for e-mail alerting - the builtin plugins are : http://circus.readthedocs.org/en/latest/for-ops/using-plugin...

But writing a new plugin is very easy, see : http://circus.readthedocs.org/en/latest/for-devs/writing-plu...

We would be happy to add in the built-in collection new useful plugins contributed by the community

Does anyone here know of a comparable project in Ruby?

There's one called BluePill: https://github.com/bluepill-rb/bluepill

really nice project, can't wait to use it for my own needs

Reply on news.ycombinator.com