Multiple BGP sessions with Level3 (DIA) started flapping at approx 03:00 Pacific:

Aug 30 03:05:13 rtr02 Rib: %BGP-3-NOTIFICATION: sent to neighbor 4.35.X.Y (AS 3356) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes

Aug 30 03:05:13 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old state Established event HoldTime new state Idle

Aug 30 03:07:37 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old state OpenConfirm event RecvKeepAlive new state Established

Aug 30 03:15:38 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old state Established event HoldTime new state Idle

Aug 30 03:17:15 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old state OpenConfirm event RecvKeepAlive new state Established

Aug 30 03:19:55 rtr02 Rib: %BGP-3-NOTIFICATION: sent to neighbor 4.35.X.Y+52091 (proto) 6/7 (Cease/connection collision resolution) 0 bytes

Aug 30 03:20:11 rtr02 Rib: %BGP-3-NOTIFICATION: received from neighbor 4.35.X.Y (AS 3356) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes

Aug 30 03:20:11 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old state Established event RecvNotify new state Idle

And incoming traffic from AS3356 and AS209 both dropped to very low volumes.

On Sun, Aug 30, 2020 at 5:58 AM Jason Kuehl <jason.w.kuehl@gmail.com> wrote:

Well, When I tried calling I got a fast busy, so that's nice.

On Sun, Aug 30, 2020 at 8:33 AM David Hubbard <dhubbard@dino.hostasaurus.com> wrote:

Same.  Also, as reported on outages list, what’s even worse is that they appear to be continuing to propagate advertisements from circuits whose sessions have been turned down.  I validated ours still were via a couple looking glass portals.  Down Detector shows nearly every major service provider impacted.

 

They’re not reachable so who knows if they’re even working on it.  I feel like they’ve been cutting heavily on the network ops side in recent years…

 

From: NANOG <nanog-bounces+dhubbard=dino.hostasaurus.com@nanog.org> on behalf of Drew Weaver via NANOG <nanog@nanog.org>
Reply-To: Drew Weaver <drew.weaver@thenap.com>
Date: Sunday, August 30, 2020 at 8:23 AM
To: "nanog@nanog.org" <nanog@nanog.org>
Subject: Centurylink having a bad morning?

 

Hello,

 

Woke up this morning to a bunch of reports of issues with connectivity had to shut down some Level3/CTL connections to get it to return to normal.

 

As of right now their support portal won’t load: https://www.centurylink.com/business/login/

 

Just wondering what others are seeing.

 

--

Sincerely,
 
Jason W Kuehl
Cell 920-419-8983
jason.w.kuehl@gmail.com