Google Docs Cloud Outage

As other Cloud Services, Google provide a status dashboard. During the 9th of september, we could notice on this dashboard that Google Docs (Google Document Lists, Google Documents, Google Drawings and Google Apps Scripts) was totally down. Google immediatly wrote:

“We’re aware of a problem with Google Docs List affecting a majority of users. The affected users are unable to access Google Docs List.”

On Google Blog, Alan Warren, the Google Engineering Director, explain why such a outage happened and why application were inaccessible for the majority of Google Apps users:

Every time a Google Doc is modified, a machine looks up the servers that need to be updated. Due to the memory management bug, the lookup machines didn’t recycle their memory properly after each lookup, causing them to eventually run out of memory and restart. While they restarted, their load was picked up by the remaining lookup machines – making them run out of memory even faster. This meant that eventually the servers couldn’t properly process a large fraction of the requests to access document lists, documents, drawings, and scripts which led to the outage you saw on Wednesday.