Bell Nexxia Outtage

So while Rogers@Home is trying to dispel a class action law suit, Bell Nexxia decides it's a good time to have an outtage.

Apparently an ATM card in one of their core switches failed Thursday at about 3:00PM. I'm glad it took them over 12 hours to fix. Here's the report about the problem from our ISP:

BellNexxia experienced a network outage late yesterday afternoon which lasted
until this morning.  BellNexxia connects almost all our ADSL clients to our
POP in Ottawa or Toronto.  They also provide the link between our Ottawa and
Toronto POP.  We have yet to receive a full explaination from BellNexxia,
but from a reliable source we were told they replaced and reprogrammed six   
Cisco 9000 series routers.  We will be pressing them to find out what they
will do to add redundancy to their network to avoid such failures in the
future.

This outage had no relation to our planned downtime last night to do a
memory upgrade on our core Ottawa router.  This upgrade was successfully
done yesterday evening.

For our most of our ADSL clients, we have to rely on BellNexxia, as there
is no other single Telco or ISP leasing DSLAM ports throughout Ontario and
Quebec.

For most of our co-location and web hosting clients we were able to
manually change IP addresses and reroute traffic in order to recover
service.  Within a month we will have our network configured to automatically
reroute traffic over a backup link should such an outage occur.

Since we provide a wide range of services, the impact of this outage varied
by client according to their subscribed services.  Over the next day, I
will be sending additional notices to clients.  These notices will be
providing additional details of the outage specific to the service the
client is subscribed to.

If your concerns are not met, please contact us, or contact me directly: 
ralph@istop.com
Although our client base for all of the services we provide has grown
above 1000 in the last year, I still personally read all my email.  I
intend to continue this, as I feel it is important for clients to be
able to communicate directly with the company owner should they wish to
do so.

Ralph Doncaster
principal, IStop.com