Hello and Good Morning, Are there reports of L3 having issues this morning ? Starting at about 10:10 A Pacific, I started seeing huge drops in traffic at various sites, including San Diego, Houston, San Antonio, Charlotte, NC, Philadelphia, etc. Anyone seeing a similar behavior ?
On Mon, 01 Aug 2011 12:39:43 -0500, Khurram Khan <brokenflea@gmail.com> wrote:
Hello and Good Morning,
Are there reports of L3 having issues this morning ? Starting at about 10:10 A Pacific, I started seeing huge drops in traffic at various sites, including San Diego, Houston, San Antonio, Charlotte, NC, Philadelphia, etc. Anyone seeing a similar behavior ?
Yes we are seeing Loss from Houston To LA (not NYC to LA) dropping out in Dallas -=Tom -- Using Opera's revolutionary email client: http://www.opera.com/mail/
On Mon, Aug 1, 2011 at 10:39 AM, Khurram Khan <brokenflea@gmail.com> wrote:
Hello and Good Morning,
Are there reports of L3 having issues this morning ? Starting at about 10:10 A Pacific, I started seeing huge drops in traffic at various sites, including San Diego, Houston, San Antonio, Charlotte, NC, Philadelphia, etc. Anyone seeing a similar behavior ?
Seeing similar issues here. Traffic going over Level3 seems to be dying. -- Brent Jones brent@servuhome.net
В 11:39 -0600 на 01.08.2011 (пн), Khurram Khan написа:
Hello and Good Morning,
Are there reports of L3 having issues this morning ? Starting at about 10:10 A Pacific, I started seeing huge drops in traffic at various sites, including San Diego, Houston, San Antonio, Charlotte, NC, Philadelphia, etc. Anyone seeing a similar behavior ?
Yes, l3 in Dallas is dropping about 40% of the traffic we're sending them. -- Regards, Vasil Kolev
On Mon, 1 Aug 2011, Khurram Khan wrote:
Hello and Good Morning,
Are there reports of L3 having issues this morning ? Starting at about 10:10 A Pacific, I started seeing huge drops in traffic at various sites, including San Diego, Houston, San Antonio, Charlotte, NC, Philadelphia, etc. Anyone seeing a similar behavior ?
Yes...about the same time here...maybe a few minutes later. I was seeing traces stopping at ae-63-63.ebr3.Atlanta2.Level3.net. Things seem to be moving again. I just unshut our L3 BGP session. Interestingly, while Level3 was having problems, our local Akamai cluster was apparently not working properly. images.apple.com resolves to our local Akamai cluster, but requests there were timing out. ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On 2011-08-01, at 1:48 PM, Jon Lewis wrote:
Things seem to be moving again.
I happen to have an L3 link out of NYC, but unfortunately I don't have a list of on-net L3 prefixes in any of the reportedly affected regions, so I'm unable to provide any data from my vantage point up here. I'm sure others are in my position as well. Is there any sort of etiquette/BCP for reporting issues like this to the community? Something that might specify a method of providing information a little more specific than just specifying the affected region(s)? Maybe a list of a few affected hosts/prefixes/URLS/etc? (incidentally, images.apple.com also resolves to our local Akamai cluster)
I did finally see a Level 3 network event posted about this in their portal. Actually they list two separate ones: A routing issue failure between Dallas, TX and Los Angeles, CA is impacting IP services. Impacted for: 1 hour 29 minutes A loss of connectivity to servers in Dallas, TX, Tustin, CA, and Tokyo, Japan caused an impact to CDN services. The second one probably explains the Akamai issues one poster mentioned. -----Original Message----- From: Jason Lixfeld [mailto:jason@lixfeld.ca] Sent: Monday, August 01, 2011 2:03 PM To: nanog@nanog.org Subject: Community troubleshooting étiquette/BCP (was: L3 Issues) On 2011-08-01, at 1:48 PM, Jon Lewis wrote:
Things seem to be moving again.
I happen to have an L3 link out of NYC, but unfortunately I don't have a list of on-net L3 prefixes in any of the reportedly affected regions, so I'm unable to provide any data from my vantage point up here. I'm sure others are in my position as well. Is there any sort of etiquette/BCP for reporting issues like this to the community? Something that might specify a method of providing information a little more specific than just specifying the affected region(s)? Maybe a list of a few affected hosts/prefixes/URLS/etc? (incidentally, images.apple.com also resolves to our local Akamai cluster)
participants (7)
-
Brent Jones -
Jason Lixfeld -
Jon Lewis -
Khurram Khan -
Scott Berkman -
Thomas Donnelly -
Vasil Kolev