Underground Data Centers are Having a Moment

Critical Thinking is a weekly column on innovation in data center infrastructure design and management. More about the column and the author here.

Another week, another underground data center. At least that is how it feels.

Maltese energy services company Enemalta announced this week a €75 million ($89 million) deal with Florida-based cloud operator Streamcast Technologies.

The deal includes fit-out of a data center in part of Enemalta’s existing underground facilities. From the companies’ joint statement:

The organizations highlighted some of the key benefits of locating data centers underground including thermal insulation and physical security. The fact that Enemalta is an energy specialist suggests that energy costs will also be competitive. The pilot project will begin at the end of this year.

Mining New Capacity

But while the Enemalta deal may be the latest example of an underground data center in Europe, it is not the most interesting so far this year. That prize goes to the Lefdal Mine Datacenter (LMD) that opened near Maloy, Norway, in May 2017.

Lefdal has taken the concept of underground data centers and run with it. The facility, backed by regional investors and Norwegian power company SFE, has potential to reach capacity of 120,000 square meters (1.3 million square feet) of  data center space and more than 200MW of IT capacity. If fully utilized it would be the biggest data center in Europe.

As with other underground data centers, the organizations behind LMD – which also include Rittal and IBM – make much of the site’s physical security. However, its cooling system and access to cheap renewable energy are probably the standout features of the site.

Fjord Thinking

LMD uses seawater cooling from an adjacent fjord to enable an extremely low Power Usage Effectiveness (PUE) rating of between 1.08 and 1.15. The fact that the mine is below sea level also means that unlike some rival water-cooled facilities no energy has to be expended pumping the seawater up to the facility.

The LMD facility can support power densities up to 50kW per rack. It currently has only one hydroelectric power feed, but LMD says the supply has a statistical reliability of 99.97 percent over the last 10 years. A second independent feed is planned for next year.

Overall, LMD says construction costs per MW for the facility are 30 to 60 percent lower than a standard data center build-out in Europe. The fact that no outer shell had to be built and less physical security features were required helped keep the costs down. 

So far only the first phase of the multi-level site is operational with approximately 45MW of cooling capacity available initially. The final facility will have six levels divided into approximately 75 chambers.

Containers Underground

When it comes to actual white space, LMD customers have a couple of options. The first is a conventional data hall based on designs from Rittal and IBM. These are effectively concrete buildings with three floors of whitespace constructed inside the mine. LMD claims they will have low construction cost and each building will have up to 2,500 square meters of net data center space.

The other option is to choose a three-layered stack of containerized data centers from suppliers including Rittal. LMD says this approach enables the complex fit-out to be done outside the mine. It also makes dealing with hardware issues easier as containers can be lifted out and replaced if needed.

Initial customers include Swiss technical financing specialist Fortuitus AG, which has installed 5MW of high-performance compute (HPC) infrastructure in the underground data center, and iNNOVO Cloud, which is supporting crypto-currency hashing services.

Stefan Sickenberger, CIO at iNNOVO Cloud in a statement:

Hypoxic Fire Suppression

Ledfal doesn’t have a monopoly on underground data centers in Norway. Green Mountain also operates two colocation facilities in the country with plans to expand into other territories. Like LDM, it also makes use of seawater cooling at its underground Stavanger facility located inside an ex-Nato munitions facility buried in a hillside next to a fjord.

The Stavanger site is also innovative due its use of hypoxic fire suppression; the oxygen content of the air is reduced to about 15 percent (from 21 percent), which prevents combustion. Eliminating traditional fire suppression systems also reduces the risk of damage to IT.

The US also has its share of companies operating underground facilities, including Iron Mountain’s The Underground, which includes data centers operating 220 feet below ground in a heavily guarded former limestone mine in Boyers, Pennsylvania.

The Downsides of Down Under

Despite the efficiency and resiliency benefits of underground facilities, there are some downsides to being down in the ground. As mentioned in a previous Data Center Knowledge article, not all underground sites are suitable for conversion to data centers. For example, mines that date earlier than the 1960s may lack the structural integrity required for an operational data center. Water and dampness may also be issues.

And while sites such as LMD have solved the problem of cooling using seawater, ventilation can be significant challenge for some underground sites; approaches such as free-air cooling may not be possible or more expensive to implement. Staff may also not relish the idea of being underground for long periods.

Another potential issue is that some underground facilities – in caves, mines, or bunkers – may be in remote locations. Connectivity and latency issues may narrow the range of potential customers. That could explain why Lefdal’s initial tenants are HPC-specific.

But while some underground sites are remote, there could be a new niche for purpose-built underground sites in population-dense areas, where land is limited and real estate prices are high. For example, Data Center Knowledge recently reported that the Singaporean government has agreed to fund a research project into building underground and high-rise data centers to help meet demand for new capacity.

As demand for new edge capacity grows –driven by content distribution requirements as well as new IoT applications and devices – its likely that underground and other space-efficient designs will only become more pervasive.

Subterranean sites may be literally underground, but it seems they have also moved from the alternative to the mainstream.