Hey Gang - I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR? Thanks, Ben
Facebook seems to also be affected. -----Original Message----- From: R. Benjamin Kessler <rbk@mnsginc.com> To: nanog@nanog.org Subject: cisco.com Date: Tue, 4 Aug 2009 09:34:46 -0400 Hey Gang - I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR? Thanks, Ben -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Both work from Austin, TX. - d. On Tue, 4 Aug 2009, Alex Nderitu wrote:
Facebook seems to also be affected.
-----Original Message----- From: R. Benjamin Kessler <rbk@mnsginc.com> To: nanog@nanog.org Subject: cisco.com Date: Tue, 4 Aug 2009 09:34:46 -0400
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
-- Dominic J. Eidson "Baruk Khazad! Khazad ai-menu!" - Gimli ---------------------------------------------------------------------------- http://www.dominiceidson.com/
Seeing same issue from Chicago via Qwest and HE.
Both work from Austin, TX.
- d.
On Tue, 4 Aug 2009, Alex Nderitu wrote:
Facebook seems to also be affected.
-----Original Message----- From: R. Benjamin Kessler <rbk@mnsginc.com> To: nanog@nanog.org Subject: cisco.com Date: Tue, 4 Aug 2009 09:34:46 -0400
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
-- Dominic J. Eidson "Baruk Khazad! Khazad ai-menu!" - Gimli ---------------------------------------------------------------------------- http://www.dominiceidson.com/
-- Chris Gotstein Sr Network Engineer UP Logon/Computer Connection UP 500 N Stephenson Ave Iron Mountain, MI 49801 Phone: 906-774-4847 Fax: 906-774-0335 chris@uplogon.com
Same here via Verizon, Level3 and Comcast. Btw... all 3 resolve to the same 198.133.219.25 addr. -----Original Message----- From: Chris Gotstein [mailto:chris@uplogon.com] Sent: Tuesday, August 04, 2009 8:48 AM To: nanog@nanog.org Subject: Re: cisco.com Seeing same issue from Chicago via Qwest and HE.
Both work from Austin, TX.
- d.
On Tue, 4 Aug 2009, Alex Nderitu wrote:
Facebook seems to also be affected.
-----Original Message----- From: R. Benjamin Kessler <rbk@mnsginc.com> To: nanog@nanog.org Subject: cisco.com Date: Tue, 4 Aug 2009 09:34:46 -0400
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
-- Dominic J. Eidson "Baruk Khazad! Khazad ai-menu!" - Gimli
----------------------------------------------------------------------------
-- Chris Gotstein Sr Network Engineer UP Logon/Computer Connection UP 500 N Stephenson Ave Iron Mountain, MI 49801 Phone: 906-774-4847 Fax: 906-774-0335 chris@uplogon.com
On Aug 4, 2009, at 9:59 AM, Jason Vanick wrote:
Same here via Verizon, Level3 and Comcast.
No trouble in Virginia with either Cox Cable or Cogent.
Btw... all 3 resolve to the same 198.133.219.25 addr.
That's what I get ;; ANSWER SECTION: cisco.com. 86400 IN A 198.133.219.25 so, that sounds like routing to me. BTW, 198.133.219.26 is ping-able from here. Marshall
-----Original Message----- From: Chris Gotstein [mailto:chris@uplogon.com] Sent: Tuesday, August 04, 2009 8:48 AM To: nanog@nanog.org Subject: Re: cisco.com
Seeing same issue from Chicago via Qwest and HE.
Both work from Austin, TX.
- d.
On Tue, 4 Aug 2009, Alex Nderitu wrote:
Facebook seems to also be affected.
-----Original Message----- From: R. Benjamin Kessler <rbk@mnsginc.com> To: nanog@nanog.org Subject: cisco.com Date: Tue, 4 Aug 2009 09:34:46 -0400
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
-- Dominic J. Eidson "Baruk Khazad! Khazad ai- menu!" - Gimli
----------------------------------------------------------------------------
-- Chris Gotstein Sr Network Engineer UP Logon/Computer Connection UP 500 N Stephenson Ave Iron Mountain, MI 49801 Phone: 906-774-4847 Fax: 906-774-0335 chris@uplogon.com
We have seen the route for cisco withdrawn from 208 and 2828. Facebook seems fine Dominic J. Eidson wrote:
Both work from Austin, TX.
- d.
On Tue, 4 Aug 2009, Alex Nderitu wrote:
Facebook seems to also be affected.
-----Original Message----- From: R. Benjamin Kessler <rbk@mnsginc.com> To: nanog@nanog.org Subject: cisco.com Date: Tue, 4 Aug 2009 09:34:46 -0400
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
Missing route on Internap also. Netraft shows cisco.com went down right at 12:00GMT. http://uptime.netcraft.com/perf/graph?site=www.cisco.com On Tue, Aug 4, 2009 at 8:48 AM, sjk<sjk@sleepycatz.com> wrote:
We have seen the route for cisco withdrawn from 208 and 2828. Facebook seems fine
Dominic J. Eidson wrote:
Both work from Austin, TX.
- d.
On Tue, 4 Aug 2009, Alex Nderitu wrote:
Facebook seems to also be affected.
-----Original Message----- From: R. Benjamin Kessler <rbk@mnsginc.com> To: nanog@nanog.org Subject: cisco.com Date: Tue, 4 Aug 2009 09:34:46 -0400
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
On Tue Aug 04, 2009, Steve Rossen wrote: Route is back 08/04 13:55:57 Withdraw 198.133.219.0/24 08/04 16:04:53 Update 198.133.219.0/24 Times are CET. German
Missing route on Internap also.
Netraft shows cisco.com went down right at 12:00GMT.
http://uptime.netcraft.com/perf/graph?site=www.cisco.com
On Tue, Aug 4, 2009 at 8:48 AM, sjk<sjk@sleepycatz.com> wrote:
We have seen the route for cisco withdrawn from 208 and 2828. Facebook seems fine
Dominic J. Eidson wrote:
Both work from Austin, TX.
- d.
On Tue, 4 Aug 2009, Alex Nderitu wrote:
Facebook seems to also be affected.
-----Original Message----- From: R. Benjamin Kessler <rbk@mnsginc.com> To: nanog@nanog.org Subject: cisco.com Date: Tue, 4 Aug 2009 09:34:46 -0400
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
All Ok from France through Sprintlink and Telia sh ip bgp 198.133.219.25 Number of BGP Routes matching display condition : 2 Status codes: s suppressed, d damped, h history, * valid, > best, i internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 198.133.219.0/24 217.118.238.45 1370 120 0 1239 109 i * 198.133.219.0/24 213.248.77.181 120 0 1299 7018 109 i Cisco and facebook available -----Message d'origine----- De : Steve Rossen [mailto:steve.rossen@gmail.com] Envoyé : mardi 4 août 2009 16:03 À : nanog@nanog.org Objet : Re: cisco.com Missing route on Internap also. Netraft shows cisco.com went down right at 12:00GMT. http://uptime.netcraft.com/perf/graph?site=www.cisco.com On Tue, Aug 4, 2009 at 8:48 AM, sjk<sjk@sleepycatz.com> wrote:
We have seen the route for cisco withdrawn from 208 and 2828. Facebook seems fine
Dominic J. Eidson wrote:
Both work from Austin, TX.
- d.
On Tue, 4 Aug 2009, Alex Nderitu wrote:
Facebook seems to also be affected.
-----Original Message----- From: R. Benjamin Kessler <rbk@mnsginc.com> To: nanog@nanog.org Subject: cisco.com Date: Tue, 4 Aug 2009 09:34:46 -0400
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
No route for 198.133.219.0/24 in 22820 from our upstream (3356 and 174). -Scott W -----Original Message----- From: sjk [mailto:sjk@sleepycatz.com] Sent: Tuesday, August 04, 2009 8:49 AM To: Dominic J. Eidson Cc: nanog@nanog.org Subject: Re: cisco.com We have seen the route for cisco withdrawn from 208 and 2828. Facebook seems fine Dominic J. Eidson wrote:
Both work from Austin, TX.
- d.
On Tue, 4 Aug 2009, Alex Nderitu wrote:
Facebook seems to also be affected.
-----Original Message----- From: R. Benjamin Kessler <rbk@mnsginc.com> To: nanog@nanog.org Subject: cisco.com Date: Tue, 4 Aug 2009 09:34:46 -0400
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
On Aug 4, 2009, at 10:03 AM, Scott Wolfe wrote:
No route for 198.133.219.0/24 in 22820 from our upstream (3356 and 174).
-Scott W
Through Cogent tme$ traceroute 198.133.219.26 traceroute to 198.133.219.26 (198.133.219.26), 64 hops max, 40 byte packets 1 dmz-mct2.americafree.tv (63.105.122.1) 0.673 ms 0.394 ms 0.243 ms 2 gi0-7.na21.b002176-1.dca01.atlas.cogentco.com (38.99.206.153) 0.690 ms 0.721 ms 0.970 ms 3 te9-2.3687.mpd01.dca01.atlas.cogentco.com (38.20.43.49) 0.984 ms 0.965 ms 0.732 ms 4 vl3491.ccr02.dca01.atlas.cogentco.com (154.54.7.234) 0.976 ms 0.923 ms 0.726 ms 5 te8-3.ccr02.iad01.atlas.cogentco.com (154.54.26.134) 54.971 ms te4-3.ccr02.iad01.atlas.cogentco.com (154.54.26.138) 3.705 ms 13.960 ms 6 sl-st30-ash-0-11-3-0.sprintlink.net (144.232.8.205) 1.973 ms 2.089 ms 1.975 ms 7 sl-crs1-dc-0-13-0-0.sprintlink.net (144.232.25.12) 3.495 ms 3.043 ms 2.734 ms 8 sl-bb20-dc-3-0-0.sprintlink.net (144.232.15.10) 2.989 ms sl-crs1- rly-0-13-5-0.sprintlink.net (144.232.19.212) 5.824 ms sl-crs1- rly-0-2-0-0.sprintlink.net (144.232.19.222) 5.860 ms 9 sl-crs1-rly-0-9-0-0.sprintlink.net (144.232.20.13) 5.613 ms 5.134 ms 4.477 ms 10 sl-gw18-sj-13-0-0.sprintlink.net (144.232.3.6) 71.695 ms 71.306 ms 72.170 ms 11 144.228.44.14 (144.228.44.14) 72.156 ms 72.895 ms 71.916 ms 12 sjce-dmzbb-gw1.cisco.com (128.107.239.89) 72.154 ms 144.228.44.14 (144.228.44.14) 72.301 ms sjce-dmzbb-gw1.cisco.com (128.107.239.89) 72.329 ms 13 sjck-dmzdc-gw2-gig5-2.cisco.com (128.107.224.73) 72.422 ms sjce- dmzbb-gw1.cisco.com (128.107.239.89) 71.853 ms sjck-dmzdc-gw2- gig5-2.cisco.com (128.107.224.73) 72.173 ms 14 sjck-dmzdc-gw2-gig5-2.cisco.com (128.107.224.73) 72.393 ms * 71.648 ms 15 * * * 16 * * * 17 * * * 18 * * * Could other Sprint routes be affected ? Regards Marshall
-----Original Message----- From: sjk [mailto:sjk@sleepycatz.com] Sent: Tuesday, August 04, 2009 8:49 AM To: Dominic J. Eidson Cc: nanog@nanog.org Subject: Re: cisco.com
We have seen the route for cisco withdrawn from 208 and 2828. Facebook seems fine
Dominic J. Eidson wrote:
Both work from Austin, TX.
- d.
On Tue, 4 Aug 2009, Alex Nderitu wrote:
Facebook seems to also be affected.
-----Original Message----- From: R. Benjamin Kessler <rbk@mnsginc.com> To: nanog@nanog.org Subject: cisco.com Date: Tue, 4 Aug 2009 09:34:46 -0400
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
FACEBOOK: UP CISCO: UP LOCATION: PORTLAND, OR David Hiers CCIE (R/S, V), CISSP ADP Dealer Services 2525 SW 1st Ave. Suite 300W Portland, OR 97201 o: 503-205-4467 f: 503-402-3277 -----Original Message----- From: Scott Wolfe [mailto:scott.wolfe@cybera.net] Sent: Tuesday, August 04, 2009 7:04 AM To: nanog@nanog.org Subject: RE: cisco.com No route for 198.133.219.0/24 in 22820 from our upstream (3356 and 174). -Scott W -----Original Message----- From: sjk [mailto:sjk@sleepycatz.com] Sent: Tuesday, August 04, 2009 8:49 AM To: Dominic J. Eidson Cc: nanog@nanog.org Subject: Re: cisco.com We have seen the route for cisco withdrawn from 208 and 2828. Facebook seems fine Dominic J. Eidson wrote:
Both work from Austin, TX.
- d.
On Tue, 4 Aug 2009, Alex Nderitu wrote:
Facebook seems to also be affected.
-----Original Message----- From: R. Benjamin Kessler <rbk@mnsginc.com> To: nanog@nanog.org Subject: cisco.com Date: Tue, 4 Aug 2009 09:34:46 -0400
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system.
I now have a route to 198.133.219.0/24 Cisco.com is back up. On Tue, Aug 4, 2009 at 10:03, Scott Wolfe <scott.wolfe@cybera.net> wrote:
No route for 198.133.219.0/24 in 22820 from our upstream (3356 and 174).
-Scott W
-----Original Message----- From: sjk [mailto:sjk@sleepycatz.com] Sent: Tuesday, August 04, 2009 8:49 AM To: Dominic J. Eidson Cc: nanog@nanog.org Subject: Re: cisco.com
We have seen the route for cisco withdrawn from 208 and 2828. Facebook seems fine
Dominic J. Eidson wrote:
Both work from Austin, TX.
- d.
On Tue, 4 Aug 2009, Alex Nderitu wrote:
Facebook seems to also be affected.
-----Original Message----- From: R. Benjamin Kessler <rbk@mnsginc.com> To: nanog@nanog.org Subject: cisco.com Date: Tue, 4 Aug 2009 09:34:46 -0400
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and
ETR?
Thanks,
Ben
-- BH
Am 04.08.2009 um 15:42 schrieb Alex Nderitu:
Facebook seems to also be affected.
facebook works fine from germany
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
An error occurred while processing your request. Reference #97.520dd58.1249393745.3bb006 is #down Marc
Just watched the rviews via bgplay for the aforementioned /24, shows them converging from AT&T internet, to AT&T Worldnet, to Sprint + Globix, to AAAaaaaaaaaaah! -----Original Message----- From: Marc Manthey [mailto:marc@let.de] Sent: Tuesday, August 04, 2009 9:50 AM To: nanog@nanog.org Subject: Re: cisco.com Am 04.08.2009 um 15:42 schrieb Alex Nderitu:
Facebook seems to also be affected.
facebook works fine from germany
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
An error occurred while processing your request. Reference #97.520dd58.1249393745.3bb006 is #down Marc
Can get to facebook but not to cisco from Denver (level 3). Traces reach SJC and die in ATT net. Possibly a local data center outage / black-holing event. - Gaurav On Tue, Aug 4, 2009 at 7:50 AM, Marc Manthey <marc@let.de> wrote:
Am 04.08.2009 um 15:42 schrieb Alex Nderitu:
Facebook seems to also be affected.
facebook works fine from germany
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
An error occurred while processing your request. Reference #97.520dd58.1249393745.3bb006
is #down
Marc
Not sure the ETA but the network that the address for cisco.com resolves to (198.133.219.0/24) is no longer in BGP. -- ----------------------------------------------------- Aaron Millisor R. Benjamin Kessler wrote:
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
Well, Cisco *did* EoS/EoL BGP last week. I guess there really wasn't all that much industry traction on whatever protocol they decided to replace it with. https://puck.nether.net/pipermail/cisco-nsp/2009-July/062646.html ;p On 4-Aug-09, at 9:45 AM, Aaron Millisor wrote:
Not sure the ETA but the network that the address for cisco.com resolves to (198.133.219.0/24) is no longer in BGP.
-- ----------------------------------------------------- Aaron Millisor
R. Benjamin Kessler wrote:
Hey Gang - I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR? Thanks, Ben
Well, Cisco *did* EoS/EoL BGP last week. I guess there really wasn't all that much industry traction on whatever protocol they decided to replace it with.
https://puck.nether.net/pipermail/cisco-nsp/2009-July/062646.html
What happened could be: a) they were smoking something and indeed decided to use EIGRP rather than BGP. b) they were testing out 4 byte ASNs and had a software issue in their IOS c) someone in Cisco wanted to download a new IOS and got frustrated with their new site so he/she pulled the plug. Kudos to that brave Cisco employee. :-) -Hank
On Tue, 4 Aug 2009, hank@efes.iucc.ac.il wrote:
a) they were smoking something and indeed decided to use EIGRP rather than BGP.
b) they were testing out 4 byte ASNs and had a software issue in their IOS
c) someone in Cisco wanted to download a new IOS and got frustrated with their new site so he/she pulled the plug. Kudos to that brave Cisco employee.
I'm assuming that the outage this morning was catastrophic enough that whoever in their IT/Neteng group was working on this couldn't open a TAC case, or there were problems opening the case and then calling back into the TAC to get it escalated to priority 1. I might be half-kidding :) jms
See: https://puck.nether.net/pipermail/outages/2009-August/001386.html I do not have a route to that IP (198.133.219.25) in BGP either.. On Tue, Aug 4, 2009 at 8:34 AM, R. Benjamin Kessler<rbk@mnsginc.com> wrote:
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
On Tue Aug 04, 2009, Jon Auer wrote:
See: https://puck.nether.net/pipermail/outages/2009-August/001386.html I do not have a route to that IP (198.133.219.25) in BGP either..
Route is not longer in the routing table since (CET) 08/04 13:55:57 Withdraw 198.133.219.0/24> German
Cisco.com up again in Italy. Regards, German Martinez ha scritto:
On Tue Aug 04, 2009, Jon Auer wrote:
See: https://puck.nether.net/pipermail/outages/2009-August/001386.html I do not have a route to that IP (198.133.219.25) in BGP either..
Route is not longer in the routing table since (CET)
08/04 13:55:57 Withdraw 198.133.219.0/24>
German
Same here in Prague (various upstreams in Central Europe) MK Jon Auer napsal(a):
See: https://puck.nether.net/pipermail/outages/2009-August/001386.html I do not have a route to that IP (198.133.219.25) in BGP either..
On Tue, Aug 4, 2009 at 8:34 AM, R. Benjamin Kessler<rbk@mnsginc.com> wrote:
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
Looks like it's back. rtr-inet1#show ip bgp 198.133.219.0/24 BGP routing table entry for 198.133.219.0/24, version 4296794 Paths: (1 available, best #1, table Default-IP-Routing-Table) Advertised to update-groups: 1 6128 7132 109, (received & used) 69.74.151.237 from 69.74.151.237 (65.19.127.20) Origin IGP, localpref 100, valid, external, best rtr-inet2#show ip bgp 198.133.219.0/24 BGP routing table entry for 198.133.219.0/24, version 11588586 Paths: (2 available, best #1, table Default-IP-Routing-Table) Not advertised to any peer 6128 7132 109, (received & used) 129.77.19.1 from 129.77.19.1 (129.77.9.252) Origin IGP, metric 0, localpref 100, valid, internal, best 6395 3356 7018 109, (received & used) 67.96.160.189 from 67.96.160.189 (216.140.10.58) Origin IGP, metric 6, localpref 100, valid, external Community: 6395:1 6395:1006 ---- Matthew Huff | One Manhattanville Rd OTA Management LLC | Purchase, NY 10577 http://www.ox.com | Phone: 914-460-4039 aim: matthewbhuff | Fax: 914-460-4139
-----Original Message----- From: Michal Krsek [mailto:michal@krsek.cz] Sent: Tuesday, August 04, 2009 9:59 AM To: Jon Auer Cc: nanog@nanog.org Subject: Re: cisco.com
Same here in Prague (various upstreams in Central Europe)
MK
Jon Auer napsal(a):
See: https://puck.nether.net/pipermail/outages/2009- August/001386.html I do not have a route to that IP (198.133.219.25) in BGP either..
On Tue, Aug 4, 2009 at 8:34 AM, R. Benjamin Kessler<rbk@mnsginc.com> wrote:
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
On Aug 4, 2009, at 10:14 AM, Matthew Huff wrote:
Looks like it's back.
rtr-inet1#show ip bgp 198.133.219.0/24 BGP routing table entry for 198.133.219.0/24, version 4296794 Paths: (1 available, best #1, table Default-IP-Routing-Table) Advertised to update-groups: 1 6128 7132 109, (received & used) 69.74.151.237 from 69.74.151.237 (65.19.127.20) Origin IGP, localpref 100, valid, external, best
rtr-inet2#show ip bgp 198.133.219.0/24 BGP routing table entry for 198.133.219.0/24, version 11588586 Paths: (2 available, best #1, table Default-IP-Routing-Table) Not advertised to any peer 6128 7132 109, (received & used) 129.77.19.1 from 129.77.19.1 (129.77.9.252) Origin IGP, metric 0, localpref 100, valid, internal, best 6395 3356 7018 109, (received & used) 67.96.160.189 from 67.96.160.189 (216.140.10.58) Origin IGP, metric 6, localpref 100, valid, external Community: 6395:1 6395:1006
Work from here as well sl-gw39-nyc-12-0-0-si28.sprintlink.net (144.228.178.109) AS/21986
http://blogs.cisco.com/news/comments/final_update_ciscocom_outage/ FINAL UPDATE: Cisco.com Outage Service to Cisco.com <http://www.cisco.com> has been restored and all applications are now fully operational. The issue occurred during preventative maintenance of one of our data centers when a human error caused an electrical overload on the systems. This caused Cisco.com <http://www.cisco.com> and other applications to go down. Because of the severity of the overload, the redundancy measures in some of the applications and power systems were impacted as well, though the system did shut down as designed to protect the people and the equipment. As a result, no data were lost and no one was injured. Cisco has plans already in process to add additional redundancies to increase the resilience of these systems. Again, we thank our customers and our partners for their patience during the resolution of this issue. Posted by <http://blogs.cisco.com/authors/bio/46> Cisco PR at 12:00AM PST ---- Matthew Huff | One Manhattanville Rd OTA Management LLC | Purchase, NY 10577 http:// <http://www.otaotr.com/> www.ox.com | Phone: 914-460-4039 aim: matthewbhuff | Fax: 914-460-4139 From: Myke Lyons [mailto:Myke.Lyons@cmtww.com] Sent: Tuesday, August 04, 2009 10:47 AM To: Matthew Huff Cc: 'nanog@nanog.org' Subject: Re: cisco.com On Aug 4, 2009, at 10:14 AM, Matthew Huff wrote: Looks like it's back. rtr-inet1#show ip bgp 198.133.219.0/24 BGP routing table entry for 198.133.219.0/24, version 4296794 Paths: (1 available, best #1, table Default-IP-Routing-Table) Advertised to update-groups: 1 6128 7132 109, (received & used) 69.74.151.237 from 69.74.151.237 (65.19.127.20) Origin IGP, localpref 100, valid, external, best rtr-inet2#show ip bgp 198.133.219.0/24 BGP routing table entry for 198.133.219.0/24, version 11588586 Paths: (2 available, best #1, table Default-IP-Routing-Table) Not advertised to any peer 6128 7132 109, (received & used) 129.77.19.1 from 129.77.19.1 (129.77.9.252) Origin IGP, metric 0, localpref 100, valid, internal, best 6395 3356 7018 109, (received & used) 67.96.160.189 from 67.96.160.189 (216.140.10.58) Origin IGP, metric 6, localpref 100, valid, external Community: 6395:1 6395:1006 Work from here as well sl-gw39-nyc-12-0-0-si28.sprintlink.net (144.228.178.109) AS/21986 [1]http://blogs.cisco.com/news/comments/final_update_ciscocom_outage/ FINAL UPDATE: Cisco.com Outage Service to [2]Cisco.com has been restored and all applications are now fully operational. The issue occurred during preventative maintenance of one of our data centers when a human error caused an electrical overload on the systems. This caused [3]Cisco.com and other applications to go down. Because of the severity of the overload, the redundancy measures in some of the applications and power systems were impacted as well, though the system did shut down as designed to protect the people and the equipment. As a result, no data were lost and no one was injured. Cisco has plans already in process to add additional redundancies to increase the resilience of these systems. Again, we thank our customers and our partners for their patience during the resolution of this issue. Posted by [4]Cisco PR at 12:00AM PST ---- Matthew Huff | One Manhattanville Rd OTA Management LLC | Purchase, NY 10577 http://[5]www.ox.com | Phone: 914-460-4039 aim: matthewbhuff | Fax: 914-460-4139 [cid:image001.jpg@01CA14F5.53540A00] From: Myke Lyons [mailto:Myke.Lyons@cmtww.com] Sent: Tuesday, August 04, 2009 10:47 AM To: Matthew Huff Cc: 'nanog@nanog.org' Subject: Re: cisco.com On Aug 4, 2009, at 10:14 AM, Matthew Huff wrote: Looks like it's back. rtr-inet1#show ip bgp 198.133.219.0/24 BGP routing table entry for 198.133.219.0/24, version 4296794 Paths: (1 available, best #1, table Default-IP-Routing-Table) Advertised to update-groups: 1 6128 7132 109, (received & used) 69.74.151.237 from 69.74.151.237 (65.19.127.20) Origin IGP, localpref 100, valid, external, best rtr-inet2#show ip bgp 198.133.219.0/24 BGP routing table entry for 198.133.219.0/24, version 11588586 Paths: (2 available, best #1, table Default-IP-Routing-Table) Not advertised to any peer 6128 7132 109, (received & used) 129.77.19.1 from 129.77.19.1 (129.77.9.252) Origin IGP, metric 0, localpref 100, valid, internal, best 6395 3356 7018 109, (received & used) 67.96.160.189 from 67.96.160.189 (216.140.10.58) Origin IGP, metric 6, localpref 100, valid, external Community: 6395:1 6395:1006 Work from here as well sl-gw39-nyc-12-0-0-si28.sprintlink.net (144.228.178.109) AS/21986 References 1. http://blogs.cisco.com/news/comments/final_update_ciscocom_outage/ 2. http://www.cisco.com/ 3. http://www.cisco.com/ 4. http://blogs.cisco.com/authors/bio/46 5. http://www.otaotr.com/
can't get to cisco.com from here atm either, but can get to facebook. looks like facebook is now coming from ashburn, va. cisco dies within level3 for us, and for route-views.oregon-ix.net: 5 eugn-core1-gw.nero.net (207.98.64.161) [AS 3701] !H * !H don't see that address (198.133.219.25) in the global routing table either. show ip bgp 198.133.219.0 % Network not in table R. Benjamin Kessler wrote:
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
I'm reaching them from Time-Warner in Portland, Maine. 1 <1 ms <1 ms <1 ms 10.0.1.10 2 1 ms <1 ms <1 ms rrcs-24-39-42-66.nys.biz.rr.com [24.39.42.66] 3 3 ms 1 ms 1 ms rrcs-24-39-42-65.nys.biz.rr.com [24.39.42.65] 4 5 ms 3 ms 4 ms ten1-1-1.ptldmecny-pe01.nyroc.rr.com [24.24.21.145] 5 10 ms 9 ms 8 ms ten2-1-1.albynyqby-pe01.nyroc.rr.com [24.24.21.133] 6 11 ms 10 ms 10 ms ten1-2-1.albynywav-pe01.nyroc.rr.com [24.24.21.46] 7 11 ms 13 ms 10 ms ge-7-2-0.albynywav-rtr03.nyroc.rr.com [24.24.21.158] 8 15 ms 14 ms 14 ms ae-5-0.cr0.nyc30.tbone.rr.com [66.109.6.74] 9 15 ms 14 ms 14 ms ae-1-0.pr0.nyc30.tbone.rr.com [66.109.6.161] 10 15 ms 14 ms 14 ms ex1-tg2-0.eqnwnj.sbcglobal.net [151.164.250.245] 11 90 ms 89 ms 89 ms ded4-g1-3-0.sntc01.pbi.net [151.164.41.161] 12 90 ms 89 ms 89 ms Cisco-Systems-1152786.cust-rtr.pacbell.net [64.161.0.62] 13 89 ms 89 ms 89 ms sjc5-dmzbb-gw1-gig1-47.cisco.com [128.107.224.105] 14 319 ms 305 ms 298 ms sjce-dmzbb-gw1.cisco.com [128.107.224.2] 15 90 ms 89 ms 90 ms sjck-dmzdc-gw1-gig5-2.cisco.com [128.107.224.69] 16 * ^C H:\> Shon Elliott wrote:
can't get to cisco.com from here atm either, but can get to facebook. looks like facebook is now coming from ashburn, va.
cisco dies within level3 for us, and for route-views.oregon-ix.net:
5 eugn-core1-gw.nero.net (207.98.64.161) [AS 3701] !H * !H
don't see that address (198.133.219.25) in the global routing table either.
show ip bgp 198.133.219.0 % Network not in table
R. Benjamin Kessler wrote:
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Thanks,
Ben
On Tue, Aug 04, 2009 at 09:34:46AM -0400, R. Benjamin Kessler wrote:
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
Instead of the hoot-n-holler line, maybe check bgp? route-views.oregon-ix.net>sho ip bgp 198.133.219.25 BGP routing table entry for 198.133.219.0/24, version 8654975 Paths: (1 available, best #1, table Default-IP-Routing-Table) Not advertised to any peer 4826 7018 109 114.31.199.1 from 114.31.199.1 (114.31.199.1) Origin IGP, localpref 100, valid, external, best route-views.oregon-ix.net> route-server>sho ip bgp 198.133.219.0/24 BGP routing table entry for 198.133.219.0/24, version 11641505 Paths: (15 available, best #2, table Default-IP-Routing-Table) Flag: 0x24A0 Not advertised to any peer 7018 109, (received & used) 12.123.134.124 from 12.123.134.124 (12.123.134.124) Origin IGP, localpref 100, valid, external Community: 7018:2000 7018:32112 7018 109, (received & used) 12.123.1.236 from 12.123.1.236 (12.123.1.236) Origin IGP, localpref 100, valid, external, best Community: 7018:2000 7018:32112 7018 109, (received & used) 12.123.21.243 from 12.123.21.243 (12.123.21.243) Origin IGP, localpref 100, valid, external Community: 7018:2000 7018:32112 7018 109, (received & used) 12.123.13.241 from 12.123.13.241 (12.123.13.241) Origin IGP, localpref 100, valid, external Community: 7018:2000 7018:32112 7018 109, (received & used) 12.123.9.241 from 12.123.9.241 (12.123.9.241) Origin IGP, localpref 100, valid, external Community: 7018:2000 7018:32112 7018 109, (received & used) 12.123.145.124 from 12.123.145.124 (12.123.145.124) Origin IGP, localpref 100, valid, external Community: 7018:2000 7018:32112 7018 109, (received & used) 12.123.139.124 from 12.123.139.124 (12.123.139.124) Origin IGP, localpref 100, valid, external Community: 7018:2000 7018:32112 7018 109, (received & used) 12.123.142.124 from 12.123.142.124 (12.123.142.124) Origin IGP, localpref 100, valid, external Community: 7018:2000 7018:32112 7018 109, (received & used) 12.123.17.244 from 12.123.17.244 (12.123.17.244) Origin IGP, localpref 100, valid, external Community: 7018:2000 7018:32112 7018 109, (received & used) 12.123.5.240 from 12.123.5.240 (12.123.5.240) Origin IGP, localpref 100, valid, external Community: 7018:2000 7018:32112 7018 109, (received & used) 12.123.137.124 from 12.123.137.124 (12.123.137.124) Origin IGP, localpref 100, valid, external Community: 7018:2000 7018:32112 7018 109, (received & used) 12.123.33.249 from 12.123.33.249 (12.123.33.249) Origin IGP, localpref 100, valid, external Community: 7018:2000 7018:32112 7018 109, (received & used) 12.123.25.245 from 12.123.25.245 (12.123.25.245) Origin IGP, localpref 100, valid, external Community: 7018:2000 7018:32112 7018 109, (received & used) 12.123.45.252 from 12.123.45.252 (12.123.45.252) Origin IGP, localpref 100, valid, external Community: 7018:2000 7018:32112 7018 109, (received & used) 12.122.125.4 from 12.122.125.4 (12.122.125.4) Origin IGP, metric 411, localpref 100, valid, external route-server> http://onestepconsulting.org/communities/as7018/ doesn't indicate anything meaningful for those communities, so presumably they are internal. 109 appears to be properly originating, so ... -- RSUC / GweepNet / Spunk / FnB / Usenix / SAGE
Cisco.com (198.133.219.25) is alive from .au (from ASN7474) Traceroute shows: 9 448 ms 419 ms 389 ms sjck-dmzbb-gw1.cisco.com [128.107.224.6] 10 427 ms 268 ms 279 ms sjck-dmzdc-gw2-gig5-1.cisco.com [128.107.224.77] Did a quick check on a few .au looking glass sites and getting entries for 198.133.219.0/24. --jay -----Original Message----- From: R. Benjamin Kessler [mailto:rbk@mnsginc.com] Sent: Tuesday, 4 August 2009 11:35 PM To: nanog@nanog.org Subject: cisco.com Hey Gang - I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR? Thanks, Ben
Cisco is aware of the issue and they are working on it. David Hiers CCIE (R/S, V), CISSP ADP Dealer Services 2525 SW 1st Ave. Suite 300W Portland, OR 97201 o: 503-205-4467 f: 503-402-3277 -----Original Message----- From: R. Benjamin Kessler [mailto:rbk@mnsginc.com] Sent: Tuesday, August 04, 2009 6:35 AM To: nanog@nanog.org Subject: cisco.com Hey Gang - I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR? Thanks, Ben This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system.
On 5/08/2009, at 1:34 AM, R. Benjamin Kessler wrote:
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
CCNAs everywhere panic as their monitoring tools tell them that the 'Internet' is down. -- Nathan Ward
2009/8/5 Nathan Ward <nanog@daork.net>
On 5/08/2009, at 1:34 AM, R. Benjamin Kessler wrote:
Hey Gang -
I'm unable to get to cisco.com from multiple places on the 'net (including downforeveryoneorjustme.com); any ideas on the cause and ETR?
CCNAs everywhere panic as their monitoring tools tell them that the 'Internet' is down.
-- Nathan Ward
Oh the hilarity
participants (34)
-
Aaron Millisor -
Alex Nderitu -
Alexander Harrowell -
Armin -
Bruce Horth -
Chris Gotstein -
Christopher Lenton -
Curtis Maurand -
Dominic J. Eidson -
Gaurav Taparia -
German Martinez -
Giuseppe Spanò - Videopiù Srl -
hank＠efes.iucc.ac.il -
Hiers, David -
Jason Lixfeld -
Jason Vanick -
Jay Mitchell -
Joe Provo -
Jon Auer -
Jonathan Bayles -
Justin Krejci -
Justin M. Streiner -
Marc Manthey -
Marshall Eubanks -
Matthew Huff -
Michal Krsek -
Moriniaux Michel -
Myke Lyons -
Nathan Ward -
R. Benjamin Kessler -
Scott Wolfe -
Shon Elliott -
sjk -
Steve Rossen