On Oct 24, 2013, at 2:13 AM, nanog-request@nanog.org wrote:

Message: 7 Date: Wed, 23 Oct 2013 22:40:34 -0400 From: JRC NOC <nospam-nanog@jensenresearch.com> To: nanog@nanog.org Subject: BGP failure analysis and recommendations Message-ID: <5.1.0.14.0.20131023214304.0396ead0@authsmtp.jensenresearch.com> Content-Type: text/plain; charset="us-ascii"; format=flowed

Hello Nanog -

On Saturday, October 19th at about 13:00 UTC we experienced an IP failure at one of our sites in the New York area. It was apparently a widespread outage on the East coast, but I haven't seen it discussed here.

We are multihomed, using EBGP to three (diverse) upstream providers. One provider experienced a hardware failure in a core component at one POP. Regrettably, during the outage our BGP session remained active and we continued receiving full routes from the affected AS. And our prefixes continued to be advertised at their border. However basically none of the traffic between those prefixes over that provider was delivered. The bogus routes stayed up for hours. We shutdown the BGP peering session when the nature of the problem became clear. This was effective. I believe that all customer BGP routes were similarly affected, including those belonging to some large regional networks and corporations. I have raised the questions below with the provider but haven't received any information or advice.

Did you provider provide an official written RFO yet? Courtney Smith courtneysmith@comcast.net () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments