27 Oct 2013

Cloud failure temporarily crashes HealthCare.gov

Posted by iwgcr

On October 27th HealthCare.gov, an American healthcare insurance marketplace, went down because of a network failure at Verizon’s Terremark cloud service. Following the recovery HHS spokeswoman Joanne Peters said in a statement:

Verizon Terremark successfully resolved the issue with the networking component overnight, and as of 7 a.m. ET this morning the Data Services Hub was fully operational. The HealthCare.gov technical team continued troubleshooting one issue with the online account creation process in the application and has now opened the online application and enrollment tools back up to consumers.

The outage was the latest in a series of technical glitches for the site, including long waits for service for users and problems in delivering information to insurance carriers on the back end.

However, this was the first problem caused by a cloud service provider.

The failure originated in a Verizon Terremark data center and likely affected other companies that purchase computing power from the tech giant, though Verizon has thus far not responded to media inquiries on the cause of the problem. It is also unclear how many potential customers attempted to reach the site while it was down only to receive a message that said: “We are experiencing technical difficulties and hope to have them resolved soon. Please try again later.”

The outage affected both the Federally Facilitated Exchange which serves insurance customers in 36 states, and the individual exchanges in 14 states and the District of Columbia. The state-based health insurance exchanges as well as the federal system, are dependent on the data hub to operate properly.

Later on, it was announced The Department of Health and Human Services signed a contract with Hewlett-PackardCo. to replace Verizon Communications Inc.’s Terremark subsidiary as its web-hosting provider for the federal health-insurance marketplace.

Date

Service

Duration

Critical Data Lost

2013-10-27 Verizon Terremark 16 hours no

 

Resources:

http://fcw.com/articles/2013/10/28/cloud-failure-crashes-healthcare-gov.aspx

http://online.wsj.com/news/articles/SB10001424052702303562904579224491970912988