Also saw a major traffic drop. There is a Root Cause to be issued early in the week I'm told.


-jim

On Fri, Jun 11, 2021 at 2:42 PM Siyuan Miao <aveline@misaka.io> wrote:
Yea, it was down but both RS are online and feeding us unreachable nexthops during the outage .

On Sat, Jun 12, 2021 at 1:27 AM Seth Mattinen <sethm@rollernet.us> wrote:
On 6/11/21 10:16 AM, Jon Lewis wrote:
> On Fri, 11 Jun 2021, Seth Mattinen wrote:
>
>> Did Any2 LAX barf last night between about 1am and 8am Pacific time?
>
> More like 00:00-7:45 (Pacific time).
>
> Anyone know what broke, and why the IX was dead for nearly 8 hours?
> This is our second recent issue with "an Any2 IX", having dealt with an
> IX partition event at Any2 Denver just a few weeks ago.
>


What I saw was a lot of unreachable nexthops (I'm in LA2) on routes
advertised through the route servers. Most of my direct BGP sessions
were down, but a handful were still working including the route servers.

For example, I was getting routes for AS29791 from the route servers,
but nexthop 206.72.211.106 was dead to me. Not to pick on Internap other
than a mutual customer called me directly at 1am and wanted to know why
things were down.

I killed the route server sessions and went back to sleep.

Feels like LA1 and LA2 got split, but however the route servers
interconnect still worked, which was problematic.