Mesosphere Raises $10.5M to Commercialize Twitter 'Fail Whale' Killer Mesos

Mesosphere, a startup with technology that aims to centralize management of IT resources across multiple data centers and provider cloud environments, closed a $10.5 million Series A funding round led by Andreessen Horowitz.

The company is commercializing Apache Mesos, which lets organizations manage a mix of distributed infrastructure resources like a single machine. Mesos powers Twitter’s private data centers and is widely credited for eliminating the “Fail Whale”, Twitter’s fail screen that was ingrained in the cultural zeitgeist during Twitter’s formative shaky-uptime years. Mesosphere will provide enterprise products based on the open source technology as well as commercial support.

Making management of an entire data center seem like management of a single gigantic computer is widely touted as the next-generation approach to data center management. It would enable resource-starved organizations scale their data centers like tech giants, such as Google or Twitter, scale theirs.

'The holy grail of cloud computing'

“Managing your data center as if it’s a single computer is the holy grail of cloud computing, and Mesosphere actually delivers on that compelling vision,” Brad Silverberg, a Mesosphere investor and former top Microsoft executive who led the company’s Windows, Office and Internet platform business units, said. “Mesos is the first technology that actually executes the vision.”

Mesosphere co-founder and CEO Florian Leibert said companies that try to scale as efficiently as Google usually get stuck because they do not have enough engineering muscle or orchestration and automation tools sophisticated enough to pull it off. “The most efficient path to cloud computing is to run on top of Mesos, and Mesosphere is unlocking that opportunity and making it more applicable for mainstream enterprise developer and ops teams.”

Google has recently taken its data center management approach a step forward, bringing online a "neural network," an artificial intelligence system that uses the massive amount of IT operations data generated by its data centers to make decisions that will ultimately drive even greater efficiency of its infrastructure.

Mesos helps optimize utilization of data center resources, automate IT operations and make application deployment easier. Mesosphere aims to enable enterprises scale applications across private data centers and cloud.

Mesos provides developers with simple command-line and API access to compute clusters for deploying and scaling applications. It solves the base-layer “plumbing” problems required to build distributed applications and makes applications portable, so they can run on different cluster environments, including private or public cloud, without rewriting code.

For operations teams, it abstracts and automates even difficult low-level tasks related to deploying and managing services, virtual machines and containers in scale-out cloud and data center environments. Mesosphere claims two- or three-fold resource utilization improvements and promises built-in fault tolerance, reduced system administration time and more predictable, reliable and efficient scaling.

Examples of companies using Mesos include Airbnb and HubSpot, both of whom use it to manage their Amazon cloud infrastructures, as well as eBay, Netflix, OpenTable, PayPal and Shopify, among others.

From Twitter and Airbnb to Mesosphere

Some of the big-name users of Mesos were not only early adopters but also  Mesos started as an open source project at University of California, Berkeley's AMPLab, where it was created by then graduate student Benjamin Hindman.

Mesosphere's Leibert founded his company after having helped Twitter and Airbnbn stand their Mesos environments up.

Twitter discovered the open source project, deployed Mesos in production and has been running its entire application infrastructure on top of it ever since, and Leibert was one of the architects that brought it to Twitter. He then went on to build a Mesos-based analytics infrastructure for Airbnb, the hyper-growth company that disrupted the hotel industry, before founding Mesosphere.

"We're using Mesos to manage cluster resources for most of our infrastructure,” said Brenden Matthews, distributed computing engineer at Airbnb. “We run Cronos, Storm and Hadoop on top of Mesos in order to process petabytes of data."

Mesosphere’s seed funding round included $2.25 million from Andreessen Horowitz, Kleiner Perkins, Foundation Capital and SV Angel. The company has built tools for making Mesos easier to use and deploy, including Marathon for large-scale orchestration and Deimos for Docker (a container solution) integration.