31 Oct 2012

US Cloud Computing survive Sandy : Why ? How ?

Posted by iwgcr

Hurricane Sandy was a late-season tropical cyclone that affected the Greater Antilles, the Bahamas, the Eastern United States, and Eastern Canada. Sandy was the largest Atlantic hurricane in diameter on record; its winds stretched approximately 1,100 miles from end to end.
– Wikipedia Hurricane Sandy

150 datacenters are located between Virginia, New Jersey and New York States. What was the impact of Sandy on those infrastructure?

We can notice than flooding from Hurricane Sandy has hobbled two data center buildings in Lower Manhattan, taking out diesel fuel pumps used to refuel generators.

Customers of Datagram were knocked offline Monday evening, knocking out high-traffic sites including Gawker, Gizmodo, Buzzfeed and Mediate.

Atlantic Metro reports that several of its data centers are offline due to “significant flooding” in its generator room.

There were also multiple reports of downtime for some tenants at 111 8th Avenue, a major communications hub which is owned by Google.

In New Jersey, situation was much better, where many facilities switched to generator power but remained online.

If this hurricane was one of the most important that US East Coast have known, most of infrastructure of Cloud Computing have resisted to this disaster although a large number of website have been offline.

The alert was given several days before the arrival of the Sandy, area which has taken exceptional measures to prepare for a disaster recovery like:

  • increase staffing at each of datacenter locations within the path of the hurricane
  • refuel all electrictal generator to their maximum, and put on standby fuel vendors
  • addition of portable pumps and generators to facilities
  • building bracing
  • backing up data to off-site facilities to disaster recovery specialists
  • moving critical instance to the west coast infrastructure
  • adding automate traffic routing in case of failure

With all these measures, the impact on the network was rather minimal. Hope, for the increase of Cloud Computing resiliency, that most disaster recovery practices will be permanent… The next disaster may occur without warning.