Gandi IWI Blog - Maintenance

Maintenance Gandi.Net and API

Leland Vandervort — Tue, 12 Jan 2010 16:18:00 +0100

In order to deploy the new version of the website, we will be performing maintenance tomorrow morning (13 January) between 06:00 and 08:00 CEST.

During this time the Gandi website and the reseller APIs will be inaccessible. It will not be possible, therefore, to perform any operations to purchase, renew or modify Gandi products during this time. This maintenance should last less than half an hour but we we are of course leaving sufficient margin in case a rollback may be necessary.

The VPS Server, Gandi Mail, Blogs, Gandi Sitemaker, and other Gandi services will not be affected by this maintenance.

We apologise in advance for any inconvenience.

Gandi Technical Team

Incident in One of Our Datacentres

Leland Vandervort — Fri, 08 Jan 2010 09:14:00 +0100

We have detected a partial outage in one of our datacenters affecting a number racks. Our technical teams are investigating the issue and we will keep you informed as further details become available.

Several services are impacted and customers may experience service degradation with the Gandi Website, Gandi Mail, and some transactional activities via the site.

The Hosting platform is not impacted.

We apologise for any inconvenience caused.

EDIT [10:42] : The outage is caused by a faulty network equipment. We are working on the full resolution of the problem. But Gandi services should not be degraded anymore.

Maintenance overnight 30-31 December

flyou — Thu, 24 Dec 2009 12:24:00 +0100

We will be performing a maintenance during the night of 30-31 December.

This maintenance will start at 00:00 AM CET, December the 31th and finish at 01:00 AM CET.

During this time most Gandi services will be impacted.

All operations will be supended throughout the maintenance period.

Thank you for your understanding.

EDIT : We will performing an other maintenance during the night of 29-30 December between 00:00 AM and 00:15 AM CET. Operations will be suspended during few minutes.

EDIT 30 dec. 00:25 AM__ : maintenance take more time than expected.

EDIT 31 dec. : Maintenance completed.

Mail platform maintenance overnight Friday to Saturday (4 Dec to 5 Dec) [ completed]

Leland Vandervort — Fri, 04 Dec 2009 14:06:00 +0100

Our technical team will be performing maintenance on the Gandi Mail platform overnight tonight, starting at midnight (00:00 CET), for a duration not more than one hour.

During this time, reception of emails will continue to function normally, but customers will be unable to access their mailboxes via POP, IMAP or the WebMail.

Once this maintenance has been completed, you will again be able to access your mailboxes.

This activity is being undertaken to improve the performance of our mail platform, and is the first in a series of schedule maintenance activities.

Thank you for your understanding.

Network Maintenance overnight 19-20 November

Leland Vandervort — Fri, 13 Nov 2009 16:12:00 +0100

We will be performing a major network maintenance operation during the night of 19 to 20 November in order to further improve our infrastructure.

This maintenance will commence on 20 November at 01:00 AM CET and finish at 07:00 AM CET.

During this time, three connectivity outages are expected, each lasting approximately one hour and will impact most Gandi services.

All hosting operations will be suspended throughout the maintenance period.

We apologise in advance for any inconvenience caused.

Network Switch Software Upgrade

Leland Vandervort — Tue, 10 Nov 2009 08:00:00 +0100

As a result of a minor software upgrade on one of our distribution switches between 07h20 and 07h45 CEST, customers may have noticed two brief interruptions in network connectivity for the hosting DNS and network access for some servers. The upgrade procedure was intended to be non-service-affecting following the in-service-software-upgrade mechanism. We apologise for the inconvenience and are investigating the issue with the manufacturer to ensure that any software upgrades are transparent.

Network Maintenance 21-23 October 2009 (cancelled for tonight)

Leland Vandervort — Wed, 21 Oct 2009 09:03:00 +0200

We will be performing major upgrades to our network distribution infrastructure during the nights of 21 to 23 October. During this time Gandi services will experience a number of intermittent disruptions in connectivity beginning at 1am (CEST) with maintenance activities finishing by 7am (CEST) each day. We will, nevertheless, endeavour to minimise impact to services where possible, but given the scale of the upgrades being performed, most services will encounter at least some disruption. Gandi Hosting will be the most affected during this operation, with an expected cumulative total of approximately 90 minutes of network outage. Other Gandi services will be partially affected.

We apologise in advance for any inconvenience this work may cause.

EDIT : The network upgrades are deferred until a further date.

Delays continue for some email mailboxes (resolved)

Ryan — Mon, 07 Sep 2009 18:07:00 +0200

Hello,

We are still experiencing difficulty with a certain number of GandiMail accounts. A maintenance operation began this morning to fix the problem and to return the situation to normal in the coming weeks.

In the coming months, you will see the release of a new GandiMail platform (v.2) which will provide more stability. We anticipate it being released in the last quarter of 2009.

UPDATE: 09/09/90 - we have installed additional hardware to help with the overload. We are currently migrating the most heavily-impacted customers to the new machine.

UPDATE: 10/09/09 - the platform is responding very well to the modifications we have made

In any case, please accept our apologies if you were affected by this incident.

Sincerely,
The Gandi Team

Gandi's website unavailable (resolved)

Ryan — Mon, 07 Sep 2009 14:57:00 +0200

Hello,

Our webservers' load balancers are currently offline. This has caused our website to become unavailable. Our technical team has been working on the problem since it began.

The side-effects of this outage are:

Management pages unavailable
API disconnected
Web forwarding not working

Please accept our apologies for any inconvenience this may cause, and we will continue to keep you informed of changes in the situation.

The Gandi Team

UPDATE: 14:51 - The situation has returned to normal

Delays with some email accounts

Ryan — Thu, 03 Sep 2009 17:49:00 +0200

We have experienced a problem on one of our filers, mainly due to a load problem. We have taken all the necessary actions to fix it, and so the situation will progressively return to normal.

Only a small number of our customers are impacted by this incident.

Incident on Air Conditioning in Datacentres - Resolved

Leland Vandervort — Wed, 19 Aug 2009 23:29:00 +0200

As a result of a problem with the air conditioning in two of our datacentre locations, we have had to halt all services in that location. This impacts pretty much a quarter of our hosting and all sitemaker services as well as one of our DNS clusters. We are working with our colocation partners to attempt to restart the services as quickly as possible.

We will keep you informed of developments on this issue.

We apologise for any inconvenience.

23:40: Our colocation partner is currently migrating to a backup cooling system.

00:40: The temperature is slowly dropping, and we will being by reactivating part of the services. Our colocation partner is still working on repairing the cooling system, however, but we do not yet have an estimated time to repair.

07:00: We are now staring to bring services back up in the second datacentre affected by the cooling system failure. Some services still degraded, however.

07:45: Most services are now available.

08:15: Incident has been resolved. Any customers still experiencing problems with their servers, please contact support via the online form.

Gandiblog : Maintenance running

flyou — Thu, 06 Aug 2009 00:00:00 +0200

A scheduled maintenance is running right now : http://iwi.gandibar.net/post/2009/07/31/Gandiblog-%3A-Maintenance

We apologize for the inconvenience of this disruption.

EDIT: 02:30 AM Services are back on line.

Gandiblog : Maintenance

flyou — Fri, 31 Jul 2009 15:00:00 +0200

A maintenance will be performed Thursday 5 August between 00:00 AM to 04:00 AM CEST.

During this period gandiblog service will be unreachable.

We apologize for the inconvenience of this disruption.

Hosting Storage scheduled maintenance

flyou — Thu, 23 Jul 2009 18:14:00 +0200

The maintenance will begin :

Start : 02:00 AM CEST Duration : 30 min

Only customers who have received an email are concerned.

We remind you that during this maintenance, which will last 30 minutes, your server will not be stopped. We will nonetheless suspend read/write operations on the disk.

If you use your disks intensively, we recommend that you stop your server, or remove the concerned disks before the maintenance begins.

Please accept our apologies for any inconvenience this may cause.

edit 02:25 Operation has been completed successfully. Affected disks should be functional now.

Partial Hosting Network Disruption

Leland Vandervort — Thu, 09 Jul 2009 17:54:00 +0200

As a result of a temporary issue on a piece of network equipment, a small portion of our hosting customers may have experienced a brief disruption to the public network connectivity for their servers. This was purely a routing issue for a few minutes. There will have been no impact to the storage network as this issue only affected the public IP LAN for some of the servers.

We apologise for any inconvenience this may have caused.

Explanation of the outage that lasted 4 hours

Nicolas — Thu, 02 Jul 2009 19:12:00 +0200

Dear Customers,

As some of you may have noticed, we experienced a power outage from 11:30 AM to 3:30 PM (CET) today. We apologise for the inconvenience caused.

The company which manages the datacenter, that houses one of our server rooms performed maintenance this morning on some of their electrical equipment. Unfortunately, a human error by the provider undertaking the maintenance led to a total blackout of the datacenter for several minutes. The outage impacted many major companies on the internet (Dailymotion, Skyblog, Pixmania, etc.), as well as a portion of our customers.

Our system automatically transfers data to another server room in the event of a problem and it was working normally until the threshold was reached too quickly. There are several reasons for this: a massive outage on a scale seldom seen, combined with our choice to build-up the other server locations gradually rather than all at once, a server rebooting process that still requires perfecting and the success of our free server operation...in short, a chain of events caused this outage for some of you.

It goes without saying that we will examine what we could have been done at our end to prevent this, for example, a quicker building up of the other server locations (the hosting operation helps this enormously), and the continued optimization of the rebooting script (in the event that the electricity gods or the datacenter's subcontractor are still out to get us).

Additionally, we will ensure we refund you for today, and are at your disposal for any additional requests you may have. We will be adding one day to the expiration date for all of the affected hosting resources throughout the day.

Please accept our sincere apologies for this inconvenience. We are working to ensure that our infrastructure is built-up in a way which assures good performance and excellent management, so that ALL of you can benefit from our technology.

Technical note: In the event of a Gandi AI server, if your server responds but one of your services (web, ftp, etc.) is not working correctly, you can fix this by simply stopping and starting your server from Gandi's website. In the event that you have an expert server, we recommend that you activate your account's emergency console and execute a fsck command on your disks.

click here to comment this news

Maintenance on Gandi Network Infrastructure

Leland Vandervort — Wed, 01 Jul 2009 13:53:00 +0200

We are performing some minor maintenance on the access and distribution layers of the Gandi Network. During this time, users may experience slight interruptions of connectivity for the hosting services, but these should last only at most a minute or so for the network to restabilise.

We apologise for any inconvenience.

Update: This maintenance is now completed. (15:15 CEST)

Late mail delivery for 27% of email mailboxes

Ryan — Tue, 30 Jun 2009 16:11:00 +0200

Following a minor incident on the GandiMail platform, 27% of the email mailboxes have be experiencing late delivery (5h maximum). No emails will be lost.

update 05h15 : the delay on the mail delivery is now resorbed.

Please accept our apologies for any inconvience this may have caused.

Customer Service Disruption [resolved]

Ryan — Mon, 08 Jun 2009 10:41:00 +0200

Hello, Due to a problem with the fiber optic network at our headquarters, the customer service department is unavailable until further notice. Please accept our apologies for any inconvenience this may cause.

UPDATEThe support team once more has access to all of its tools, though has a large backlog to recover. We are doing everything possible to return to an acceptable quality of service.

Incident on the hosting.

aegiap — Sat, 06 Jun 2009 11:15:00 +0200

Due to a network equipment failure, some virtual machines are now unreachable. Our staff is currently working on fixing this issue. We apologise for any inconvenience.

12:15 AM : All maintenance are completed.