preillyme

So what do you think of things like Apache Mesos in this regard?

I for one would love some help with making our docs a bit better for sure.

So not anymore than I can do with Mesos and Docker on other cloud providers. I can use Marathon for a nice UI as well.

The idea of being dox'ed is so scary to me.

Seriously, "What does it feel like to have a meat log in your pants 24/7?". Wow.

Engineers love challenges, and no challenge is more ambitious or ridiculous than building an operating system at the scale of the datacenter.

The main idea behind Mesosphere has always been to allow developers to treat a data center like a single computer — with Mesos and other software packages abstracting much of the basic devops work away. Some companies that currently use Mesos are Twitter and Airbnb.

As Ericsson begins to move downstream and directly compete with tech companies like IBM and HP, the next few years are going to be bloody interesting in that trillion-dollar intersection of telecom, IT and media.

Thanks for sharing. I really like Consul, by HashiCorp, it provides a really cool distributed key–value store and service discovery layer.

I'm looking forward to accessing Dropbox files from the Office app and saving new files to Dropbox.

Marathon is a Mesos framework for long-running "always on" apps that can serve up http among other things.

It ensures that an app stays up even when machines or entire racks fail.

Schedule:

6:30 - 7:00 Networking and Light Dinner

7:00 - 7:10 Announcements

7:10 - 7:40 Mesosphere: Deploying Docker Containers at Scale with Mesos and Marathon

Abstract: The norm these days is to operate apps at web scale. But that’s out of reach for most companies. Marathon is an open source project that makes it considerably easier to deploy Docker containers at web-scale using Apache Mesos.

See how Mesos and Marathon can help deploy and manage Docker containers at scale and how the Mesos cluster scheduler builds highly-available, fault-tolerant web-scale apps.

Bio: Sunil studied Computer Science at the University of Cambridge, where he wrote his dissertation on delay-tolerant networking systems. After four months spent cycling across Africa he returned to London to join a major investment bank and then Last.fm as a Data Engineer. Last year his interest in robotics brought him to UC Berkeley, where he completed a Master of Engineering degree with a focus on distributed computing and unmanned aerial vehicles, working in his spare time on an UAV startup. After leaving Cal, he joined Mesosphere as a Distributed Applications Engineer and is currently working on easy deployment of Mesosphere clusters on Google Cloud Platform.

7:40 - 8:10 Airbnb: How we're using Mesos at Airbnb today

Abstract: Mesos is the operating system for Airbnb's data platform. We use Chronos, Marathon, Hadoop, and other frameworks for operating a complex system of data warehouse and analytics tools. In this talk, we'll discuss how some of the day-to-day operational problems are solved using these systems.

Bio: Brenden Matthews is a software engineer at Airbnb, working on the data infrastructure team. He is an Apache Mesos contributor, the primary maintainer for Chronos and Hadoop on Mesos, and has also contributed to related frameworks such as Storm, Spark and Marathon. Previously, he has given talks at about Mesos at Twitter, QCon, and MesosCon.

8:10 - 8:30 Ericsson: On the use of Mesos and Marathon for Multi-Datacenter Service Orchestration

Abstract: With the availability of numerous geographically distributed datacenters, there is an increasing need for highly distributed service orchestration. Apache Mesos and Marathon are valuable starting points for further exploration towards a system enabling such service orchestration while achieving high datacenter utilization and providing high availability. However, both frameworks are designed and used for (resp. on) a single datacenter.

In this talk I will present prospectives on leveraging Mesos and Marathon as building blocks to orchestrate Docker-based services across multiple datacenters.

Bio: Eugen is a Senior Research Engineer at Ericsson Research Silicon Valley Lab. He currently builds systems which leverage real-time big-data analytics for improved resource management in large-scale datacenters. Prior to that, he was a postdoc in the Data Intensive Systems Group of the Advanced Computing for Science Department at the Lawrence Berkeley National Lab (LBNL). He holds a PhD in Computer Science from the University of Rennes 1, France which he received while working in the Myriads INRIA project-team. During his PhD he worked on systems and algorithms for autonomic and energy-efficient resource management. Eugen received his Bachelor and Master of Science degrees in Computer Science from the Heinrich Heine University Duesseldorf (HHU), Germany. He was a student research assistant in the HHU operating systems group, interned in the Myriads INRIA project team and was a summer intern at the LBNL.

8:30 - 9:00 Common Q&A

Yeah, I'm for sure missing launchd and memcached. But I suppose they've got their reasons.

How so? What would you like to see handled differently?

Docker explicitly violates the principles of the Twelve-Factor App. Docker apps don’t rely on any external environment. In fact, Docker demands that you store all config values, dependencies, everything inside of the container itself. Apps communicate with the rest of the world via ports and via Docker itself. The trade-off is that apps become a little bit bulkier (though not significantly), but the benefit is apps become maximally portable.

In essence, Docker makes almost no assumptions about the app’s next home. Docker apps care about where they are even less than twelve-factor apps. They can be passed to and fro across servers—and, more importantly, across virtualization platforms—and everything needed to run them (besides the OS) comes along for the ride.

Is using gorilla/context really the only easy way to store an retrieve data that is specific to the current HTTP request? Seems like there should be a better way to map values and retrieve them later from a global mutex on a map of request objects, or something.

The pthread_atfork() function shall declare fork handlers to be called before and after fork(), in the context of the thread that called fork(). The prepare fork handler shall be called before fork() processing commences. The parent fork handle shall be called after fork() processing completes in the parent process. The child fork handler shall be called after fork() processing completes in the child process. If no handling is desired at one or more of these three points, the corresponding fork handler address(es) may be set to NULL.

The order of calls to pthread_atfork() is significant. The parent and child fork handlers shall be called in the order in which they were established by calls to pthread_atfork(). The prepare fork handlers shall be called in the opposite order.

I'm not sure if that's the best approach, but it's an attempt at least.

Yeah I'm excited to read about indexes in PostgreSQL. Indexes also add overhead to the database system as a whole, so they should be used sensibly — I'm looking forward to seeing your approach for good index candidates.

Oh it's really cool that these tools support linking with OpenSSL to provide SSL support.

Well I'm happy about the volume bug fixes. That's a good step in the right direction.

I really love the quote, "The grass may be greener, but you better believe the water bill is a lot higher!"

I'm not so sure that Andreessen is what’s wrong with corporate America. He does amazing work with a great deal of amazing companies. I think that Carl Icahn is dead wrong in this respect.

Wow, when doing argument reduction from the 80-bit long-double format there will be almost complete cancellation? Damn.

"mesos tail -f task stdout stderr" FTW

Thanks for providing a link to the GitHub repository. What's the status of the docs in https://github.com/mesosphere/mesos-cli/tree/master/docs

I'm really excited about the lazy matching. It's great to see because Task IDs are long and normally require cut/paste to get the exact matches. Now that task parameters are partial matches there's no need to type that long thing in.

I'm really not sure where I stand regarding limited availability vs. eventual consistency. But, I do know that I like that Orly aims between these two extremes.

Could Cayley use Orly for a storage engine?

Has anybody tried out Orly? https://github.com/orlyatomics/orly

So legally how can @AOL claim that @creativecommons Attribution License [CC-BY] licensed content can't be used by @PeoplePlus http://wrd.cm/HudPS7...? I'm confused.

https://twitter.com/preillyme/status/397771582367477760