Oath, Verizon’s Big Bet on Digital Media, is Expanding Data Center Capacity

Five months ago, when Verizon closed its $4.5 billion acquisition of Yahoo, along with the digital media properties came a substantial data center footprint that supports them and a large team with deep data center design and operations expertise.

The same team that’s built out 10s of megawatts of data center capacity across three US campuses and one in Singapore – in recent years using the innovative, hyper-efficient Yahoo Computing Coop design – is now supporting all 50 brands under the umbrella of Oath, the digital media subsidiary Verizon created as it sealed the deal. In addition to Yahoo properties, such as Yahoo Finance and Yahoo Sports, Oath controls brands that became part of Verizon in 2015, when it spent $4.4 billion to buy AOL. Among many others, they include TechCrunch, HuffPost, Engadget, and AOL itself.

The team that ran AOL data center infrastructure has been merged with Yahoo’s team, and this integrated team now manages AOL and Yahoo data centers and other IT assets as a single infrastructure that supports Oath. The man in charge of it is Mike Coleman, the former senior director of data center operations at Yahoo, who is now VP of Infrastructure as a Service for Oath.

In addition to the former Yahoo data center campuses in Quincy, Washington; Omaha, Nebraska; Lockport, New York; and Singapore, Oath’s infrastructure includes the AOL campus in Manassas, Virginia, called the Manassas Tech Center, footprint in numerous colocation facilities (the company has multiple content delivery networks, which typically consist of relatively small infrastructure deployments in many data centers around the world), and applications running in all major public clouds, Coleman said in an exclusive interview with Data Center Knowledge.

Valuable Data Center Talent

The 15 percent staff reduction that was expected to come as part of melding Yahoo and AOL together didn’t include the data center team. “There were no layoffs in the physical infrastructure group at all,” Coleman said.

Oath is a big bet for Verizon, which like other companies that own and operate huge networks is trying to carve out a place for itself in the digital media market. Since they own the pipes, goes the thinking, these companies should have a natural advantage over content companies in building a vertical play that includes both the pipes and the content that’s delivered over those pipes. Infrastructure that stores and delivers the content is critical to the success of this big bet, which means Oath needs all the data center talent it can afford.

And the talent it has in the former Yahoo data center team is not trivial. In addition to the people who design, build, and operate the Computing Coop data centers, the team includes OpenStack engineers who built the cloud that supports most of Yahoo’s customer-facing products and engineers who design custom servers and racks that are crucial to making the coop design work.

Building Large in Quincy

Verizon obviously expects Oath to grow. The current goal is for its brands to reach 2 billion consumers by 2020 – up from about 1 billion at the time of the acquisition. To support a growing user base, a media company of this scale usually needs to build more data centers, and Coleman’s team has been busy doing just that.

The company recently launched a new data center in Quincy, Washington, home to one of the former Yahoo data center campuses and to a large Microsoft data center cluster. Using a refined version of the Computing Coop design, Oath has added 25MW of capacity in what is only the first phase of expansion at the site – already much more capacity than there was previously. When completed, the expansion will reach about 40MW of critical IT capacity total, Coleman said.

Refined Coop Design Delivers Major Density Bump

And all that power is now packed in a much smaller footprint than would have been required had the design not been refined. The design is so energy efficient (a rare PUE of 1.08, according to Coleman) because it doesn’t rely on energy guzzling mechanical chillers to cool water; nor does it rely on more efficient ways to cool water, such as Facebook’s evaporative cooling system (basically an enormous swamp cooler). In fact, it doesn’t use water for cooling at all.

Borrowing an idea from chicken coops, which are shaped in a way that takes advantage of little else besides the laws of physics to make outside air move through the space, the data center has a pitched roof with a cupola at the top. By maintaining slightly positive air pressure inside, the facility takes outside air in through massive louvers along its walls; the air flows through the servers, gets warmer, and rises to the top to be expelled through cupola louvers. Unlike most data centers, where massive electric air handlers deliver air cooled by chilled water to the servers, the Computing Coop relies on server fans to take cool air from the space that surrounds them. No air is forced to server air intakes. A fan wall helps maintain positive air pressure inside the space, ensuring that enough fresh air comes in to replace the air that’s been expelled. (You can learn about the design in detail in this paper, published by the US Department of Energy’s Office of Scientific and Technical Information.)

Aerial view of the Oath data center campus in Quincy. The building on the right is the recently launched 25MW expansion.

Simply by making the cupola larger and by increasing surface area of the cupola louvers, the data center design team was able to substantially increase the amount of power the system can cool in a single rack. “We’ve increased density by 50 percent with just some minor tweaks to the height of the cupola, so it can capture and expel more air,” Coleman said. To get a 50 percent power density bump, cupola louver height was increased from 9 feet to 13 feet, increasing the louver surface area from 2,520 square feet to 3,640 square feet (the louvers remained 280 feet wide).

Learn how Oath's data center design team was able to increase power density by 50 percent in detail from Mike Coleman directly, when he presents at the upcoming Data Center World Global conference in San Antonio on March 12-15, 2018.

The new facility can support 20kW per rack. (Average power density in traditional enterprise data centers is somewhere around 5kW per rack.) Not all applications need this kind of density, but the amount of those that do is rising, he said. Modern applications increasingly run on high-density computing equipment housed in smaller physical footprint.

Design Patents Have a New Owner

Oath continues to use the Computing Coop data center design, but it no longer owns the patents that describe it. Verizon didn’t buy all Yahoo assets. The internet company’s 15 percent stake in the Chinese e-commerce and cloud giant Alibaba, its 35.5 percent stake in Yahoo Japan, and a massive repository of of intellectual property contained in thousands of patents were all put in a publicly traded holding company called Altaba, created specifically to own the Yahoo assets Verizon didn’t buy. The deal was structured in a way that would allow Oath to continue using the intellectual property in perpetuity, Mike Sefanov, an Oath spokesman, said.

If Verizon’s big bet on digital media works out the way the company hopes it will, that intellectual property will get a lot of use. While Oath isn’t running at the same scale the hyper-scale cloud platforms are running at, its infrastructure strategy is similar to theirs. The company is building data centers at massive scale, using its own custom hardware, and optimizing for maximum efficiency. According to Coleman, the new streaming TV service Oath has in the works isn’t likely to have much of an impact on its infrastructure strategy, but if the new company’s business strategy gains traction, you can at the very least expect more Computing Coops to go up in the future.