9 Nov 2012

Salesforce: three outages on three consecutive days

Posted by iwgcr

Salesforce engineers were busy early this week.

On Sunday 11/04, during a planned maintenance at 11:00 UTC on the NA2 instance, the salesforce.com Technology Team has discovered an unexpected locking of accounts in the database tier  that caused subsequent tasks to miss their scheduled execution time.
Salesforce does not indicate which consequence such blocking was for users or how long it was effective. All that we kwow, is:

The issue delayed the scheduled completion of the planned maintenance by 48 minutes

The next two days, it was the CS3 instance that was in trouble. Monday an human error caused an interruption of communication between the CS3 database and storage tiers for 1 hour and 20 minutes. Tuesday, for 20 minutes, the instance has:

an inability to access the Salesforce application

Without more information on Salesforce System Status Dashboard, we can’t explain why.

Date

Service

Duration

Critical Data Lost

2012-11-04 Salesforce 1 hours no
2012-11-05 Salesforce 1 hours no
2012-11-06 Salesforce 20 minutes no