Hacker News Reader

Have you checked out DCOS? Once you have your cluster set up, your path to installing Spark is running "dcos package install spark" on your laptop.

View on HN

Huh, no I haven't. Although the package support appears to be more narrow than with Kubernetes/Docker, but DCOS is a possibility too.

You should also check out Ambari, fantastic open source Hadoop/spark cluster management tool.

Second the motion. Ambari is actually one of the key elements of the tech stack we're using for our service. With the "Blueprints" feature and the REST API, it's a really convenient way to automate the provisioning of Spark/Hadoop clusters.

Even better, Ambari isn't actually limited to installing just Hadoop/Spark, etc. In principle, you could extend it to take care of installing pretty much anything.

Well, the included packages in the "dcos package" CLI repo are just the prepackaged frameworks. You can actually deploy Kubernetes on top of DCOS in the same way - it natively supports Docker containers as an execution method.

Reply on news.ycombinator.com