Based on the data we’ve, it looks like there was a noticeable spike in packet loss within the AWS's Ashburn and Ohio datacenter around 17:50-18:10 Central Time. It is possible that this had something to do with the drops and dips in traffic you listed. If you’d like some additional data, feel free to email me off list. Disclaimer : I work for ThousandEyes. -Deepak On Tue, May 1, 2018 at 8:28 AM, Mike Hammett <nanog@ics-il.net> wrote:
This is going to be extremely scientific. </sarcasm>
Did anyone else see "weird" stuff going on about 10 hours ago, about 6 PM Central on April 30th? I'm based in the Chicago area and have a large client on the east coast. I saw about a 25% drop in traffic for a half hour to an hour. Another Midwestern ISP also saw about 25% drops on two different upstream connections. A friend of mine runs an ISP in Virginia. He reported a dip in traffic (though didn't report how much of a drop), but also pings and IPSEC\L2TP worked, but couldn't SSH or do other activities. That friend reported another ISP in Virginia had problems at that same time. An ISP in Cyprus reported issues at that time. I'm looking to firm up what kind of issues and verify the time.
Yet other ISPs report no problems at all. Smooth ramps on traffic graphs as one would expect at the beginning of prime time.
I thought maybe "something" happened in Ashburn (fiber cut, DWDM card failure, etc.) as my client has a wave from the east coast to me in Chicago, but then the Midwestern ISP shouldn't have any dependency on Ashburn, given Chicago and Dallas. Then there's the guy in Cyprus, which shouldn't have any bearing on anything that happens over here in the States.
I thought maybe it was an epic failure at one of the CDNs or other content networks, but then that wouldn't impact SSH or other management activities.
Anyone else have any other data to help figure out what caused this?
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest-IX http://www.midwest-ix.com
-- *~Deepak*