Continuous Deployment with Mesos, Marathon and Docker

mesosphere.com

38 points·ssk2··5 comments

Out of curiosity, who is using Mesos in production?

At GoGuardian, we've got ~10 offline workers, ~10 web services, and ~30 batch/cron jobs and we're starting to see pain in resource utilization. Is anyone running Mesos at a similar scale?

My next project is to take a look at it and perhaps make a trial system using it. Please tell me about your pain points.

We're using Opsworks/Chef for managing deployments and server configuration. I hate the resulting tight coupling between applications and the servers that they run on. In some cases, we have great utilization (where the application needs to auto-scale based on load and exceeds the capacity of a single box). However, in many other cases, we have significant underutilization (where the application uses a tiny fraction of the box's capacity).

Multi-tenancy (multiple apps on a single box) could alleviate our underutilization. It is possible with Opsworks/Chef, but it isn't a first-class feature.

Finally, we have a rather sad story for our batch/cron jobs. We have a single server and a master crontab for that server. Because of the various types of batch jobs we need to run (shell, python, java, php, node etc.), the cron server needs have runtimes for all of those.

Have you considered using Torque under Mesos for batch jobs?

Re underutilization: it's a big problem with clusters in general. Here's a decent talk on the topic:

https://www.youtube.com/watch?v=YpmElyi94AA

Here's a list: http://mesos.apache.org/documentation/latest/powered-by-meso...

Reply on news.ycombinator.com