Data center consolidation runs rampant

While it used to be common for every enterprise to have its own data center for delivering and receiving Web traffic, a new study from security vendor Arbor Networks suggests that this is no longer the case.

In its Internet Observatory report, Arbor notes that consolidation of content providers has led to the rise of “a small number of very large hosting, cloud and content providers” that generate and consume an estimated 30% of all Internet traffic. For instance, Arbor estimates that Google alone accounts for 6% of all Internet traffic in the world.

Arbor Chief Scientist Craig Labovitz says that there are several reasons for this migration of traffic from individual enterprise data centers to “hyper giants” such as Google, Facebook and Microsoft, including the rising costs and recourse demands of maintaining a data center and the aggressive efforts by large companies to buy up video, mail and other Web service companies. Additionally, he says companies that built their own data centers years ago found them quickly outdated and that they didn't have the money to properly upgrade them.

“Until a few years ago, there had been an overabundance of data centers,” he says. “The data centers built five years ago are now out of date and there are entire generations of data centers where there's no way to upgrade them.”

The solution for companies, he says, has been to consolidate their infrastructure through virtualization or to outsource many of their IT operations to the cloud.

“Starting with outsourced Web e-mail, we have seen a large migration of Web traffic out of small enterprise data centers and toward these large players,” Labovitz explains. “The cost of data centers had started to affect companies' bottom lines and that has set the stage for what's starting to happen in the transit market.”

Arbor says that another consequence of content providers and content delivery networks becoming larger has been the decreasing importance to Tier-1 transit providers such as Verizon Business, AT&T and Level 3 in delivering Web traffic. And because these companies have lost some of their profitability in the transit market, Arbor says that they've turned themselves more toward value-added services.

“Over time, IP connectivity services became indistinguishable from one provider to the next,” says Arbor in its research brief. “In response, providers started competing chiefly on price. This price competition drove down IP wholesale market prices and forced many Tier 1 networks to pursue higher-value product offerings such as CDN, cloud computing and a greater focus on private enterprise offerings.”

Arbor conducted its study of global traffic patterns by analyzing nearly 3,000 peering touers across nine Tier-1, 48 Tier-2 and 33 consumer and content providers in four different continents. Arbor said that at its peak, “the study monitored more than 12 terabits-per-second of offered load and a total of more than 256 exabytes of Internet traffic.”