Gandi IWI Blog - alert

IP Transit Outage in France Telecom

Leland Vandervort — Thu, 21 Jan 2010 12:16:00 +0100

Some customers located on networks in France Telecom (Orange, OpenTransit, etc.) may be experiencing difficulties accessing Gandi Services. A network incident has occurred within the FT network and is affecting many of their customers' access to large portions of the internet. France Telecom are working to resolve the issue.

We will keep you updated as more details are made available.

Update: 13:50 CET: The issue at France Telecom appears to have been resolved.

Datacenter outage- resolved

Nicolas — Sun, 20 Dec 2009 19:43:00 +0100

We experiment a major outage in one of our datacenter rooms. We'll keep you updated as soon as we have additional information.

Please accept our apologies for the inconvenience

Gandi team

20:26 CET it was a power outage, services are coming back now

20:55 CET Gandi's website is back online, as well as email. The GandiBlogs will return shortly as well as the servers that were affected. We are currently verifying the disks' Raid state

00:10 CET All services are now stable, however we have still received no communication from our hosting partner as to the cause of the outage. As such we are remaining on standby in case of any further incident

09:20 CET 21/12/09 Equinix have scheduled a maintenance window at 10:30 CET to switch over to generator power. There should be no impact to services, but our teams are on alert in case this activity does not go according to plan.

--11:10 CET 21/12/09__ Equinix have not yet effected the switchover. Our teams are still on standby alert while we wait for an update.

11:30 CET 21/12/09 Equinix have now effected this switchover.

Power outage at the datacenter (fixed)

Ryan — Thu, 29 Oct 2009 11:30:00 +0100

Hello,

A power outage occurred at the datacenter, TeleHouse2 this morning from 8:45 AM (Paris time) to 11:00 AM. Gandi houses several servers at that location, and they were affected by that outage, leading to disruptions in the following services:

the website http://www.gandi.net
operations requiring updates (DNS changes, addition/removal of shares, changes in forwarding addresses, etc.)
some email accounts
reseller API

Our provider acted quickly to return to a normal level of service, and at this time everything is up and running again. You may notice a delay of about an hour in the updating of your domain's DNS (if you requested that they be changed while the incident was underway), however.

This incident is unrelated to the maintenance that we had scheduled for last night, and was completely beyond our control.

Thank you for your patience and understanding.

Incident Gandi Blogs

flyou — Mon, 19 Oct 2009 15:37:00 +0200

We have encountered a disruption of services due a problem in our Blogs platform from 02:57 AM CEST to 03:18 AM CEST.

We apologize for the inconvenience

A stupid phishing attempt is underway!

Ryan — Thu, 15 Oct 2009 15:39:00 +0200

Hello,

We have heard from many of our customers that they have been receiving phishing emails.

These mails take the following form:

Dear user of the yourdomain.com mailing service!

We are informing you that because of the security upgrade of the mailing service your mailbox (you@yourdomain.com) settings were changed. In order to apply the new set of settings click on the following link:

http://yourdomain.com/owa/service_directory/settings.php?email=you@yourdomain.com%from=yourdomain.com%fromname=vous

Best regards, yourdomain.com Technical Support

The phishing scheme seems idiotic at first, because the clickable URL uses your own domain name.

In reality, however, the link is not the real link, which points to an external server, and asks you to perform a download, which seems to be a version of a known trojan called Zeus/Zbot.

The server(s) in question is/are already registered in the various anti-phishing systems.

In any case, we recommend that you continue to remain vigilant, since one stupid phishing attempt can hide another !

We will take this opportunity to remind you of the basic rules of use.

You must always verify when you enter your password that you are on Gandi's website, our SSL certificate is an EV Business certificate, and is Green in your web browser's address bar. It is not possible for an idiot internet criminal to get one of these (at least, not yet...).

Gandiblog : database problem

flyou — Thu, 16 Jul 2009 00:35:00 +0200

We are current experiencing an issue with the Gandi Blogs database.

Technical teams are currently working to solve the issue.

We apologize for any inconvenience this may cause.

EDIT : Problem was fixed at 01:47 AM CET

Gandiblog : issue

flyou — Mon, 13 Jul 2009 12:02:00 +0200

We are current experiencing an issue with the Gandi Blogs infrastructure.

We are currently working to resolve the problem.

We apologise for any inconvenience this may cause.

Update: database problem fixed

Electrical problem in one of our data centers

Ryan — Thu, 02 Jul 2009 11:39:00 +0200

A part of our hosting platform has been impacted by a total power outage in one of our data centers. We are currently working on the problem so that the situation can return to normal ASAP.

We will, of course, provide you with additional information on this incident once we have reestablished the service.

12:20 UPDATE: the power outage is over in datacenter 1. The servers are all restarting automatically, though it will take about half an hour for all of the affected servers to be rebooted, given the large number involved.

13:20 UPDATE: it will finaly take more time for some customer as the hosting backend is highly stressed. We will check that all servers will be back up at the end of the process.

15:30 UPDATE : the servers have now all been rebooted, however, some remain that need an fsck check. We are taking care of that.

Gandi SiteMaker Incident (fixed)

Ryan — Tue, 23 Jun 2009 09:52:00 +0200

An incident has occurred with on Gandi SiteMaker platform which has rendered your websites offline. We will keep you informed of the situation as soon as the problem has been identified. Please accept our apologies for any problem this may have caused. 10:38 => incident fixed.
11:00 => Following the network outage and subsequent impact on the SiteMaker service, we need to perform some emergency maintenance to bring the file system back up. It should take about an hour, in the meantime, there will be a maintenance page showing.
13:00 => Maintenance done

Network: Global Crossing IP Transit Degraded

flyou — Fri, 03 Apr 2009 04:07:00 +0200

Due to a technical problem at Global Crossing, we experienced an interruption of two of our internet transit connections. Although the traffic was transferred to our other transit connections, some users may have experienced a slight interruption to service for a few moments while internet routes reconverged.

The two connections went down at 02:48 (CET) and were restored fully operational at 04:45 (CET).

Please accept our apologies for this inconvenience.

The incident has now been resolved.

.EU Maintenance

Nicolas — Tue, 17 Feb 2009 12:17:00 +0100

EURid will perform system maintenance on Tuesday, 17 February. The maintenance will be carried out during the maintenance window, between 18:00 and 23:00, Central European Time. They expect to limit downtime to the first half hour. All Gandi services related to .EU domains (purchase, renew, update...) will not be possible during that time.

Delay in hosting operations

Ryan — Mon, 19 Jan 2009 15:20:00 +0100

Hosting operations (start/stop/reboot, machine creations, etc.) are currently suspended. Our teams are currently working on the problem which will be fixed within the hour.

16:00 CET Problem fixed

Hosting : add/renew shares beta customers

flyou — Sat, 03 Jan 2009 14:59:00 +0100

On the virtual machines, adding and removing shares are delayed. Our technical team is working to fix this issue. This incident only involve *beta customers*.

We apologize for the inconvenience.

UPDATE : The incident is closed, operations running normally at present.

GandiMail : slight disturbance

Sam — Thu, 20 Nov 2008 19:08:00 +0100

You may be experiencing slight perturbations on GandiMail (slowness of reception).

We are currently working to restore a normal flow.

We apologize for the inconvenience.

Edit : The Gandimail service is now running. All the slow emails in spool file are now delivered to your mailboxes.

Problem with a Beta filer not responding

Ryan — Thu, 30 Oct 2008 15:41:00 +0100

Filer 18 (SATA) is currently not responding. We are going to progressively shut down the 375 servers that are on this filer in order to assure that there is no data corruption.

Our technical team is currently on location and working on solving the problem. We estimate that this will be resolved within an hour.

Please accept our apologies for any inconvenience this may have caused.

UPDATE: The problem has been identified as being hardware-related. Our technicians are currently making the necessary repairs. We will keep you informed of our progress. As a reminder, this hardware is no longer used on the final release architecture.

UPDATE: We have built a new filer, and VMs are now restarting on it. We are watching your fscks, this seems okay everywhere. An overload of the filer was anticipated for the evening, though has been absorbed nicely. We will soon be migrating this filer to SAS.

Problem: Registry server down (fixed)

Ryan — Fri, 01 Aug 2008 14:43:00 +0200

Since 12:00 GMT today, one of our servers that communicates with the registries has stopped responding.

This meas that certain domain name operations are not possible. Our technicians are on location and fixing the problem. We will keep you informed as to the progress of this incident.

In the meantime, please accept our excuses for this delay in the processing of your orders.

UPDATE: Everything has returned to normal at 13:45 GMT. In the next 4 hours, we will be performing some necessary maintenance that will last a maximum of a quarter of an hour, in order to secure the other equipment.

DNS problem with french ISP.

aegiap — Thu, 24 Jul 2008 09:42:00 +0200

At the moment we are experiencing some network troubles with the free.fr DNS.

All Gandi hosting services are not reachable (using domain name) with a connectivity by free.fr.

We are working on a workaround for the problem.

EditSeems to be fixed by free.fr at 9.55 CET

Gandiblog Incident

Soraya — Sat, 05 Jul 2008 15:59:00 +0200

We have been encountering serious performance problems with GandiBlog.

Our teams are working to re-establish service as quickly as possible.

Please accept our sincere apologies.

EDIT 16:24(CET) : the problem was fixed.

Console access to hosting VM.

aegiap — Sat, 05 Jul 2008 11:22:00 +0200

The console access to hosting VM was unavailable for few hours. The service is now running.

DNS attack underway

Ryan — Mon, 30 Jun 2008 16:13:00 +0200

Hello,

A DDoS attack against our DNS is currently underway. We are in the process of countering it, though given the size of the attack, it is possible that there may be some temporary perturbations.

UPDATES

16:28 CET Attack terminated
17:00 CET New attack underway
17:42 CET New DoS attack is now terminated.

The technical team