Explanation of the outage that lasted 4 hours
By Nicolas on Thursday 2 July 2009, 19:12 - Maintenance - Permalink
Dear Customers,
As some of you may have noticed, we experienced a power outage from 11:30 AM to 3:30 PM (CET) today. We apologise for the inconvenience caused.
The company which manages the datacenter, that houses one of our server rooms performed maintenance this morning on some of their electrical equipment. Unfortunately, a human error by the provider undertaking the maintenance led to a total blackout of the datacenter for several minutes. The outage impacted many major companies on the internet (Dailymotion, Skyblog, Pixmania, etc.), as well as a portion of our customers.
Our system automatically transfers data to another server room in the event of a problem and it was working normally until the threshold was reached too quickly. There are several reasons for this: a massive outage on a scale seldom seen, combined with our choice to build-up the other server locations gradually rather than all at once, a server rebooting process that still requires perfecting and the success of our free server operation...in short, a chain of events caused this outage for some of you.
It goes without saying that we will examine what we could have been done at our end to prevent this, for example, a quicker building up of the other server locations (the hosting operation helps this enormously), and the continued optimization of the rebooting script (in the event that the electricity gods or the datacenter's subcontractor are still out to get us).
Additionally, we will ensure we refund you for today, and are at your disposal for any additional requests you may have. We will be adding one day to the expiration date for all of the affected hosting resources throughout the day.
Please accept our sincere apologies for this inconvenience. We are working to ensure that our infrastructure is built-up in a way which assures good performance and excellent management, so that ALL of you can benefit from our technology.
Technical note: In the event of a Gandi AI server, if your server responds but one of your services (web, ftp, etc.) is not working correctly, you can fix this by simply stopping and starting your server from Gandi's website. In the event that you have an expert server, we recommend that you activate your account's emergency console and execute a fsck command on your disks.
click here to comment this news
As some of you may have noticed, we experienced a power outage from 11:30 AM to 3:30 PM (CET) today. We apologise for the inconvenience caused.
The company which manages the datacenter, that houses one of our server rooms performed maintenance this morning on some of their electrical equipment. Unfortunately, a human error by the provider undertaking the maintenance led to a total blackout of the datacenter for several minutes. The outage impacted many major companies on the internet (Dailymotion, Skyblog, Pixmania, etc.), as well as a portion of our customers.
Our system automatically transfers data to another server room in the event of a problem and it was working normally until the threshold was reached too quickly. There are several reasons for this: a massive outage on a scale seldom seen, combined with our choice to build-up the other server locations gradually rather than all at once, a server rebooting process that still requires perfecting and the success of our free server operation...in short, a chain of events caused this outage for some of you.
It goes without saying that we will examine what we could have been done at our end to prevent this, for example, a quicker building up of the other server locations (the hosting operation helps this enormously), and the continued optimization of the rebooting script (in the event that the electricity gods or the datacenter's subcontractor are still out to get us).
Additionally, we will ensure we refund you for today, and are at your disposal for any additional requests you may have. We will be adding one day to the expiration date for all of the affected hosting resources throughout the day.
Please accept our sincere apologies for this inconvenience. We are working to ensure that our infrastructure is built-up in a way which assures good performance and excellent management, so that ALL of you can benefit from our technology.
Technical note: In the event of a Gandi AI server, if your server responds but one of your services (web, ftp, etc.) is not working correctly, you can fix this by simply stopping and starting your server from Gandi's website. In the event that you have an expert server, we recommend that you activate your account's emergency console and execute a fsck command on your disks.
click here to comment this news