etcd Plugin Coming to Mesosphere’s Data Center OS

Mesosphere, a startup built around Apache Mesos, open source software that makes big server clusters behave like a single computer, is working to build support for etcd, a system for maintaining consistent configuration across all nodes in a cluster, into its data center OS.

etcd was created by CoreOS, a San Francisco startup that makes software for companies that want to operate their data centers the way web-scale operators like Google and Facebook do. CoreOS announced the first stable release of etcd, also open source, today.

“Shared configuration and shared state are two very tricky domains for distributed systems developers as services no longer run on one machine but are coordinated across an entire data center,” Benjamin Hindman, chief architect at Mesosphere and chair of Apache Mesos, said in a statement. “Apache Mesos and Mesosphere’s Data Center Operating System will soon have a standard plugin to support etcd.”

Mesosphere launched an early-access preview of its data center OS in December and plans a public launch sometime in the first half of 2015.

etcd is a distributed key-value store. One of the things it helps ensure is preventing failure in a single node from bringing down the entire cluster.

Google’s open source application-container management system Kubernetes and Pivotal’s open source Platform-as-a-Service technology Cloud Foundry both use etcd. It is an important component of the Linux variant that is the chief product of CoreOS.

etcd is an implementation of Chubby, a software tool Google designed to manage consistency across server clusters in its own data centers. The big difference between the two is the “consensus protocol” each of them uses. Chubby relies on a protocol called Paxos, while etcd uses Raft. Both are written in Go.

Mesos is used by a swath of high-profile Internet companies, including Twitter, Airbnb, and Hubspot. Hindman, one of the original creators of Mesos, came to Mesosphere from Twitter, where he and his colleagues applied the software to make the social network’s infrastructure more resilient.

Mesosphere raised a $10.5 million Series A funding round in 2014.

Hindman said Mesosphere’s data center OS users have been asking for etcd support, so the company is responding to demand.

CoreOS raised $8 million of venture capital in a Series A round last year.

Toward the end of 2014 the company created some controversy by introducing an application container standard called App Container as an alternative to Docker, the highly popular open source container technology that CoreOS itself had been a big supporter of.

The company said Docker’s security model was flawed. It also accused Docker Inc., the company, of steering away from its original mission by adding container management features to what it apparently expected to remain a simple basic container.