After Kubernetes’ Victory, Its Former Rivals Change Tack

It was June 22, 2015. In just a few months’ time, Docker, Inc. had completely upended the server-based workload automation space with an extraordinary new tool. With it, a developer could build and bundle just the code a workload required to be operational. And with a handful of commands that she could enter from her couch while binge-watching Netflix, she could deploy that code to a cloud-based platform and see it virtualized and running in a matter of minutes. Just weeks before, this could have taken months.

The company was celebrating its revolution with a victory lap in the basement ballroom of the Marriott Marquis in downtown San Francisco.

“For most of computing history,” extolled the company’s CEO at the time, Ben Golub, “people thought of applications as these monolithic things that would be slow to change, that would run on a single server. And how we conceptualize building infrastructure around that, and tools around it, and virtualization was all about shrinking down that [monolith].

“But the truth is, all of those assumptions have changed. We want to build applications out of loosely-coupled services that are rapidly updating, that are deployed across many small servers. And we want to make an application that can move quickly from the laptop, through dev and testing, and into production. We want to make sure we can spread across a cluster. We want to move between clouds. And we actually want to get to the point where those many small servers include phones and cars and devices. And the applications themselves, of course, need to change.”

Related:A Hyperconvergence Progress Report: Has Kubernetes Stolen the Show?

Golub was outlining not just a new chapter for the distribution of workloads in the data center, but an entirely fresh and as-yet-unopened volume. The history of data center infrastructure from that point forward would be focused less on the server than on the workload — its portability, its flexibility, and its security integrity. Where a countless array of software vendors failed to address an obvious set of enterprise needs, a San Francisco startup that was an absolute unknown with no history, few customer contracts, and a whale as its logo had answered the call.

Google’s Suspect Timing

On that day, as a sort of gift of appreciation to the open source community that helped validate it, Docker offered its Linux-based software container packaging format to the Linux Foundation as an open source project. It became known as the Open Container Initiative (OCI).

Yet during that same week, Google made a curious move. It ceded stewardship of Kubernetes, which had been described as an orchestrator and manager for Docker containers, to another Linux Foundation project: the Cloud Native Computing Foundation (CNCF). Most of its members and most of the new container-format group’s members were the same stakeholders.

Related:What Service Meshes Are, and Why Istio Leads the Pack

Very few considered Google’s timing suspect. Those who did were perceived as troublemakers or even worse: journalists.

“All of a sudden,” I wrote at the time for The New Stack, “the stage is set for very familiar companies other than Docker and other than CoreOS to produce their own container systems without owing Docker so much as a kiss goodbye. Don’t think for a moment that IBM and VMware and Red Hat and Microsoft all signed that agreement because they have some great ideas for Docker plugins.”

Ben Golub, then CEO of Docker, Inc., speaking at DockerCon 15

Not even two years later, the market that Docker Inc. catalyzed had become known informally as “the Kubernetes ecosystem.” There were three competitors in the workload orchestration space: Kubernetes, which emerged from an internal Google project previously dubbed “Borg;” Docker Swarm, the cluster manager that had begun shipping with the commercial Docker Enterprise product; and Apache Mesos, which had the virtue of scheduling and managing server clusters for so-called “non-imaged workloads,” such as the Hadoop and Spark big data systems.

Mesosphere Showed Up for a Marathon but Lost a Sprint

Mesosphere was the startup launched in 2013 by Mesos’ creators to produce a commercial Mesos platform. It had produced a management platform called DC/OS (Data Center Operating System) and billed it as a master control system for clustered servers. It also made Marathon, a workload orchestration platform clearly intended to go toe-to-toe against Kubernetes. Mesosphere’s aim was to remake data centers as cohesive computing complexes, with a new kind of operating environment at the center. Not even two years ago, we were gauging its progress to that end.

At first, Mesos was the agent distributing workloads among its components, but soon Kubernetes wedged its way in there as well. Mesosphere’s technologists began arguing that the two schedulers could work side-by-side, or that a Kubernetes environment could become the object of a Mesos template.

Then something gave way. Microsoft’s partnership agreements with Docker Inc. had become superseded by its all-in support for Kubernetes. It adopted the orchestrator as a workload deployment platform for Azure, and even hired co-creator Brendan Burns and other prominent members of the Kubernetes community without the usually defensive Google raising the slightest fuss. Red Hat had built OpenShift into a complete, commercial Kubernetes platform. Rather than try to replicate it, IBM bought Red Hat outright in a deal valued at $34 billion, essentially redefining the mission of history’s most important computer company.

Beached

Docker Inc. continues to exist, describing itself as a developer tools company. Meanwhile, the Docker whale has become the occupant of numerous industry scrapbooks. Late last year, Docker cancelled its scheduled 2020 Austin conference, replacing it with an online broadcast. And in a move whose intent could not be interpreted as anything other than righting a wrecked train, the company sold the core product of its business, Docker Enterprise, to OpenStack and Kubernetes support firm Mirantis.

“Given our focus on developers, we are going to continue to invest in developer-facing, app-centric open source technologies like Compose, BuildKit/Buildx and Docker CLI,” Docker’s current CEO, Scott Johnston, wrote in a note to Data Center Knowledge. Compose is the tool with which a software developer builds source code and supplemental libraries into a deployable container. BuildKit is a system for automating the assembly of the components of such a container for environments where continuous integration (CI/CD) is being recognized. Docker CLI is the company’s command line tool for accessing Docker functions.

“Much more to come,” added Johnston.  “In addition, we are active in other relevant, adjacent upstream non-Docker open source projects, like Kubernetes, Cloud Native Application Bundles, and OCI.”

Mirantis bought Docker Enterprise partly because of the platform’s “revenue generating capability and potential,” Mirantis’ senior VP of marketing, David Van Everen, said in a recent interview. Those things were “obvious,” he said. One of the most prominent components of the company’s product line prior to acquiring Docker Enterprise had been its managed Kubernetes services.

If there ever was a container orchestration war, it was too short for most observers to get a good snapshot. A plurality of analysts judging the virtues of each environment believed Swarm to have been technically superior to Kubernetes for many tasks, and a greater plurality declared Swarm easier to use.

“Docker Swarm is an extremely important part of what we brought on board,” remarked Van Everen. “It’s important because it does work. I’ve had multiple Docker Enterprise customers come to me and rave about its ease of use, simplicity, and effectiveness at just working, and working reliably. We don’t want to disrupt any of that; we want to provide continuity to those Docker Swarm customers.”

Prior to Mirantis’ move, Docker Inc. had stated its intention to give Swarm a two-year sunset period before declaring it deprecated in Docker Enterprise – in favor of Kubernetes. But the Mirantis deal is sure to extend its life beyond that. “Until there’s a compelling event that is going to guide them towards our Kubernetes-based offering,” Van Everen told us, “we’re content with supporting them on Swarm.”

Docker Inc. plans to continue contributing something to the Kubernetes project. We asked Johnston whether his company will collaborate with Mirantis on Swarm or any other Docker Enterprise component, even now that Mirantis has hired that product’s core developers.

“Mirantis alone owns defining and driving the Docker Enterprise platform and roadmap. We jointly collaborate with them and others upstream in open source communities. Docker’s mission is to help developers bring their ideas to life by conquering the complexity of app development with Docker Desktop and Docker Hub,” responded Johnston, referring to his company’s development environment and container repository, respectively.

On the Third Day

The company that was Mesosphere remains healthy. But in a move obviously intended to ensure its survival in a transformed market, it changed its name last August to D2IQ (which its staffers pronounce “Day 2-IQ”). It continues to produce what it still sometimes calls “Mesosphere DC/OS.” Yet Marathon’s place in its product line is now occupied by Ksphere, a suite of components for managing and distributing containerized workloads that now includes what’s still called Mesosphere Kubernetes Engine (MKE).

“We have very robust Spark, Cassandra, and Kafka workloads running on DC/OS,” Chandler Hoisington, D2IQ’s senior VP of engineering and product, told Data Center Knowledge, referring to Apache’s open source big data analytics engine, key/value database engine, and data streaming engine, respectively. “Our customers don’t feel the Kubernetes community is quite ready to take on those production workloads.

“We’ll get there,” he added. “And I think that’s a big reason why we did the rebranding of the company. . . We don’t want to box ourselves into people thinking we just have expertise in Mesos. Yeah, Mesos was a very core part of our offerings and still is. But there’s a lot more we can help people with, leveraging the expertise we have.” The rebranding “was less about saying, ‘We’re no longer going to do DC/OS, no longer going to do Mesos.’ It was more about saying, there are a lot of other things we’re capable of helping our customers with, and the name kind of boxed us into one technology.”

If there truly are technology “ecosystems,” like planetary bodies orbiting a sun, then it’s clear they have never tolerated the presence of twin stars. Unlike soda brands on grocery store shelves, no two products in computing have ever sustained a duopoly for long. The fade-out periods for CP/M, DR DOS, and OS/2 were at least long enough for journalists to chronicle the events they were witnessing.

The benefactors of Apache Mesos and Docker Swarm insist their legacies live on. In the data center, legacies certainly do persist longer than folks anticipate. Their candles are burning in someone’s window, at least metaphorically. But in these cases particularly, the legends burned out long before their stories could even be told.