As Hurricane Ike bore down on Houston one Friday last September, the Continental Airlines' flight operations center, located on the 14th floor of a glass-sided downtown high rise, suddenly went dark. For the airline's pilots and flight crews, however, business proceeded as usual.
Here's why: At that same moment, 42 miles north of the city and some 60 feet underground — in a hardened Cold-War era bunker built by a paranoid millionaire oilman to survive a nuclear holocaust — Continental's backup data center took over. Throughout the ordeal — from Friday morning, as the storm approached, through Saturday, when winds above the Westland Bunker in Montgomery, Texas, gusted to 125 miles per hour, until Sunday evening, when operations resumed in Houston — the airline managed an 89% on time rating for its global flight schedule.
Locating a backup data center in an underground bunker may seem like overkill, even in a hurricane zone. But the facility met all of the airline's requirements — including cost, says John Stelly, managing director of technology at Continental. The bunker, run by real estate partnership Montgomery Westland, has been converted into 33,000 square feet of rack-ready data center space complete with air conditioning, redundant network and power sources, uninterruptible power supply systems and backup generators.
Continental leases 2,000 sq. feet underground and another 12,500 sq. feet of office space above ground, in a hardened building complete with 3-inch-thick bulletproof windows. The airline can house its entire operations staff of up to 125 people at the backup site.
After Hurricane Katrina, Continental began looking for a fallback data center for use during hurricanes. Westland “was far enough away to be out of harm's way but close enough for folks to drive to,” Stelly says. The blast-resistant facility is admittedly a bit much for even Continental's backup needs, but the four-feet-thick walls and high security entrance are nice extras, Stelly says.
Also, connectivity options at the Westland facility were a plus. The network and power feeds for the bunker were sourced from areas well away from Houston, while pricing was competitive with above-ground co-location facilities.
Rise of the underground
With a renewed focus on data center outsourcing and space in high availability facilities in short supply, investors such as Montgomery Westland have snapped up and renovated abandoned mines and military bunkers in the hopes of cashing in.
Since 2007, for example, Cavern Technologies has operated a data center 125 feet below ground in an abandoned limestone mine. The mined out area underground, which covers 3 million sq. feet, is 15 minutes outside of Kansas City, Mo. Unlike other mines, the Cavern facility was created with the idea of reuse in mind, so floor space isn't irregularly shaped like other underground facilities can be, says president John Clune. The area's relatively low electricity costs, at 3.5 cents per kilowatt-hour, help to make operating costs lower than those in other parts of the country, he adds.
Another facility, The Bunker, is a decommissioned, 50,000 sq. foot Royal Air Force bunker that operated until the 1990s. The facility is inside a hill near Dover, England and it now hosts data centers 100 feet below ground. “People get a picture of a hole in the ground. That's not the case. It's a state-of-the-art data center,” says Paul Lightfoot, director of managed services for The Bunker. Clients range from businesses running mission-critical Web applications to a financial services firm that runs online trading systems. “We do everything from basic square footage to fully maintained systems,” he says.
Iron Mountain is among the oldest and best known providers of underground storage and data center space. Known for storing everything from backup tapes to old movie reels in The Underground, its repurposed limestone mine in rural Pennsylvania, the company has seen its electronic storage and leased data center space business increase while its traditional paper record storage business has slowed. “It is now the fastest growing component of our business,” says vice president Charles Doughty. In addition to leasing rack-ready space, the company offers data center design, hosting and management services.
But while interest is up, the number of actual customers leasing space in its underground data centers remains small. Iron Mountain counts five operating data centers in its underground facility, including its own. But with 60,000 square feet of available data center space and another 145 acres undeveloped in the facility, Iron Mountain has plenty of room for more.
Underground data center facilities fall into two categories: Abandoned mines, like Iron Mountain's, and decommissioned military bunkers such as InfoBunker, a subterranean facility just outside of Des Moines, Iowa. InfoBunker leases data center space to organizations ranging from a local telephone company to government agencies about which Jeff Daniels, a vice president at the company, says he can't talk.
On the demand side, an increase in extreme weather events, heightened concerns about security since 9/11 and the need to provide higher levels of security to comply with regulatory requirements have made these spaces more attractive to some organizations. Underground facilities offer security and structural protections that would be cost prohibitive to build from scratch.
Meanwhile, the recession and credit crunch have made it harder to get funding to build new data centers, and organizations have become more accepting of the idea of using co-location facilities to house mission-critical data center operations. “Demand for computer space is stronger than I've ever seen it [and] the supply is so small, so inadequate,” says Peter Gross, vice president and general manager at HP Critical Facilities.
Basic co-location space isn't the problem, says Jon Bolen, chief technology officer at Westec Intelligent Surveillance. The surveillance-monitoring service, based in Des Moines Iowa, serves clients such as McDonald's Corp. and Zales Corp., and recently completed a search for a hardened facility for its own backup data center. During this search, Bolen saw a general lack of high-end infrastructure, of enterprise-class data center space. “If you need space that is as good or better than the space you would build there's a shortage of places you can go.”
This shortage has given the underground facilities an opening to pull in larger data center clients. Cavern says it is negotiating with Fortune 500 clients looking to lease spaces of 30,000 to 100,000 square feet. However, most clients are smaller organizations that don't require so much space; more typical would be a hospital that leases 1,500 sq. feet.
One might assume that IT organizations would have to pay a premium for bunker space. After all, the cost of building such a structure is high, and special venting and air-flow systems are required. But IT executives say they've driven deals where the total cost of ownership is competitive with above-ground facilities. Because they're repurposing existing space that the government or a mine operator paid to build, providers say they don't have to pass on the original construction costs for the structures and can afford to be cost competitive.
Consider the cons
Before deciding to go underground, IT executives need to identify potential limitations, experts say. Even things as simple as ceiling height can be a challenge. Continental's data center space in the Westland bunker has 10-foot ceilings, and putting full-height racks on top of an 18-inch raised floor was a tight fit. “We had to come up with a design to allow us to use full-height racks while providing sufficient airflow,” Stelly says.
Another concern: While computer systems may be protected in a bunker, critical infrastructure needed during a disaster, such as generators, fuel tanks and air conditioning cooling towers, may be above ground. That could be a problem if the catastrophe you need to worry about is a tornado, warns Westec's Bolen.
Bolen recounts how one company claimed that its hardened facility could withstand a direct hit from an F3 (158 to 206 mph) tornado. But the air conditioning and generators were outside. “When an F3 hits, those generator and HVAC units are going to come off their pads,” he says.
Westec ended up taking space at InfoBunker, about 45 miles away from its offices, Bolen says. The 65,000 square foot Cold War command bunker, designed to withstand a 20-megaton nuclear blast, maintains all infrastructure, including generators, fuel and cooling equipment, 50 feet underground.