How Deutsche Telekom Used Open Source and Automation to Build a Telco Cloud

Several years back, when Germany's Deutsche Telekom decided to upgrade its information management system for fixed-line voice services, it decided to go all-in and replace the system it had with a "next-generation IMS" (NIMS) implementation of its own design, based on DevOps automation and cloud-native technology designed for the telco cloud.

This was no small feat. Deutsche Telekom is the largest telco provider in Europe by revenue, and when the new system is completely rolled out it will manage up to 80 million subscribers' voice calls.

Earlier this month, at Red Hat Summit, Deutsche Telekom's group head of voice and messaging (DevOps) Christoph Hilz was on hand to talk about the project in an "Ask the Experts" session. Joining him were Azhar Sayeed, Red Hat's chief technologist of cloud and virtualization, and Mathias Kokot, VP of cloud-ready data center at Juniper Networks. (The two vendors participated in the project.)

"Deutsche Telekom is known for its high-quality approach, and that is especially valid for voice," Hilz said. "Being number one in all benchmark tests is always our clear ambition and with that mindset we started to replace our existing platform two to three years ago. We wanted to set up something completely new which reflects our ambition, and NIMS is multivendor, disaggregated, open, fully automated, and, of course, it's cloudified."

Getting Rid of the Old

Until this project, Hilz said, Deutsche Telekom had built its systems much like IT systems have been built since the beginning of the data center: with proprietary solutions (both hardware and software) fraught with vendor lock-in and built-in limitations.

"We had the hardware and the software from one vendor," he said. "They were all blackbox silos, and there was no synergies between the silos from supplier A and supplier B."

"The challenge at that point was we had vendor lock-in, we had long delivery times, and we were not efficient due to a low degree of automation," he added.

It was around that time, Hilz said, that the company devised a set of goals for itself that it calls the "NIMS three-two-one-zero vision," which set a goal of three months to roll out a feature(compared to the typical one to two years in a traditional IMS environment), two days from the time new software from a vendor is released before it's deployed, one day to implement a new bug or security patch, and zero "nightshifts" (dedicated time when a server or system is taken offline for scheduled maintenance).

"We knew that was a moonshot," he said.

"Brutal" Automation

It was obvious to Hilz and to his partners that nearly total automation (what he "brutal automation") would be needed if the company was to realize its goals.

"I think it was [during] the time when we created our DevOps organization that we really had problems with efficiency in the lab and later on, when it came to the rollout in the operational area," he said. "It was always our idea to have an end-to-end automation chain running, without any breaks in between, so that we can use a function onboarding the software in the lab, then use the same function for deploying the software."

The project's heavy reliance on end-to-end automation has worked out well for Deutsche Telekom, according to Hilz.

"We see that it works," he said. "We have multiple sites already implemented, so the cloud infrastructure is up and running. We already have close to one million subscribers migrated to the platform."

"We also have a huge number of interconnection partners already migrated, which is quite sensitive traffic," he added. "It's voice, so you are in real time. Voice is not good if you have packet loss or drops, because there is no retransmission for voice. We've been live for a few months now and are increasing our migration levels."

Open Source's Role

Hilz said it would have been impossible for the project to meet its goals without the use of open source software.

"Open source is the one essential part for bringing us where we are with NIMS," he said.

In addition to eliminating the prospect of vendor lock-in, which was a major goal from the start, he said that open source brought many other advantages to the table, citing its mainstream use, which means vendors and developers both know how to navigate the open source landscape.

He added that open source also fit the quality approach that the company wanted to take.

"We needed to make sure that we selected partners like Juniper and Red Hat, who provide enterprise support for open source software like OpenStack," he said. "We really felt comfortable going in that direction."

Realizing the Vision

Which brought Hilz back to the three-two-one-zero vision.

"I said from the beginning that it was a moonshot," he said. "I think we are seeing that we are coming closer and closer, day by day, because that's the North Star for the team. The question is, will the vision three-two-one-zero always stay a vision, or will we be able to transform it to reality?"