Network Solutions has decided to filter our CIDR addr, 169.132 / 16 since about 5 PM yesterday from reaching the internic. Below is a forwarded copy of email exchange between myself, and MCI -- It appears that there is no backup routing engineer at the Internic which troubles me greatly. MCI has been attempting to contact them every 30 minutes since yesterday to no avail. This is the second time in a month or so that this has happened. Since I was unable to contact Mark Kosters, I felt it might be helpful to contact Kim Hubbard - To my dismay, she refused to acknowledge our problem and hung up on me with an attitude. <sigh> Of course, this pleases me to no end. ---------- Forwarded message ---------- Date: Mon, 19 Aug 1996 17:11:32 -0400 From: Nour Elouali <nour@mci.net> To: markk@netsole.com, allwync@rs.internic.net Cc: bnite@tremere.ios.com, trouble@mci.net Subject: Access Denied Hi there, One of our peers _idt.net_ cannot reach your site, see the forwarded traceroute.. They cannot route to you from any IP Address within their CIDR Block 169.132.0.0/16. According to my traceroute, I see that they get filtered out at network-solutions which is your provider. Will you be able to assist? Thanks... -Nour Elouali MCI NOC ****** border7.wtn#tr 198.41.0.6 Type escape sequence to abort. Tracing the route to rs1.internic.net (198.41.0.6) 1 network-solutions.Washington.mci.net (204.70.77.42) 8 msec 8 msec 4 msec 2 rs1.internic.net (198.41.0.6) [AS 6245] 8 msec 4 msec 4 msec ****** ------- Forwarded Message Delivery-Date: Mon, 19 Aug 1996 17:27:36 -0500 Received: from postoffice.Reston.mci.net by carbon.cary.mci.net (8.6.12/kaw-mci.net/feb95) id NAA02326; Mon, 19 Aug 1996 13:27:24 -0400 Received: from tremere.ios.com (tremere.ios.com [169.132.32.250]) by postoffice.Reston.mci.net (8.7.5/8.7.3) with ESMTP id NAA28560 for <trouble@mci.net>; Mon, 19 Aug 1996 13:27:21 -0400 (EDT) Received: (from bnite@localhost) by tremere.ios.com (8.7.5/8.6.9) id NAA08739 for trouble@mci.net; Mon, 19 Aug 1996 13:27:11 -0400 (EDT) Date: Mon, 19 Aug 1996 13:27:11 -0400 (EDT) From: Golan Ben-Oni <bnite@tremere.ios.com> Message-Id: <199608191727.NAA08739@tremere.ios.com> To: trouble@mci.net Subject: Traceroute to Internic traceroute rs.internic.net traceroute to rs.internic.net (198.41.0.6), 30 hops max, 40 byte packets 1 eth-3.gw-4.hck.idt.net (169.132.32.254) 2.907 ms 2.026 ms 3.068 ms 2 eth-0.gw-1.hck.idt.net (169.132.34.250) 107.054 ms 121.335 ms 77.012 ms 3 10.255.255.1 (10.255.255.1) 82.342 ms 70.764 ms * 4 192.41.177.181 (192.41.177.181) 55.270 ms 48.858 ms 42.972 ms 5 core2-hssi2-0.Washington.mci.net (204.70.1.213) 40.422 ms 40.915 ms 43.822 ms 6 border7-fddi-0.Washington.mci.net (204.70.74.51) 33.906 ms 78.689 ms 75.278 ms 7 * * * 8 * * * 9 * * * 10 * * * 11 * * * 12 * * * 13 * * * 14 * * * ------- End of Forwarded Message
Network Solutions has decided to filter our CIDR addr, 169.132 / 16 since about 5 PM yesterday from reaching the internic. Below is a forwarded copy of email exchange between myself, and MCI --
It was my understanding that Network Solutions was not a provider, but a customer of several providers, including MCI. Your statements and the attached note from MCI seem to indicate otherwise. Or did this particular outage have anything to do with the MCI instabilities around KC yesterday? Or perhaps with your use of net10 in the path?
---------- Forwarded message ---------- Date: Mon, 19 Aug 1996 17:11:32 -0400 From: Nour Elouali <nour@mci.net> To: markk@netsole.com, allwync@rs.internic.net Cc: bnite@tremere.ios.com, trouble@mci.net Subject: Access Denied
Hi there,
One of our peers _idt.net_ cannot reach your site, see the forwarded traceroute.. They cannot route to you from any IP Address within their CIDR Block 169.132.0.0/16.
According to my traceroute, I see that they get filtered out at network-solutions which is your provider. Will you be able to assist? Thanks...
-Nour Elouali MCI NOC
******
border7.wtn#tr 198.41.0.6
Type escape sequence to abort. Tracing the route to rs1.internic.net (198.41.0.6)
1 network-solutions.Washington.mci.net (204.70.77.42) 8 msec 8 msec 4 msec 2 rs1.internic.net (198.41.0.6) [AS 6245] 8 msec 4 msec 4 msec ******
------- Forwarded Message
Delivery-Date: Mon, 19 Aug 1996 17:27:36 -0500 Received: from postoffice.Reston.mci.net by carbon.cary.mci.net (8.6.12/kaw-mci.net/feb95) id NAA02326; Mon, 19 Aug 1996 13:27:24 -0400 Received: from tremere.ios.com (tremere.ios.com [169.132.32.250]) by postoffice.Reston.mci.net (8.7.5/8.7.3) with ESMTP id NAA28560 for <trouble@mci.net>; Mon, 19 Aug 1996 13:27:21 -0400 (EDT) Received: (from bnite@localhost) by tremere.ios.com (8.7.5/8.6.9) id NAA08739 for trouble@mci.net; Mon, 19 Aug 1996 13:27:11 -0400 (EDT) Date: Mon, 19 Aug 1996 13:27:11 -0400 (EDT) From: Golan Ben-Oni <bnite@tremere.ios.com> Message-Id: <199608191727.NAA08739@tremere.ios.com> To: trouble@mci.net Subject: Traceroute to Internic
traceroute rs.internic.net traceroute to rs.internic.net (198.41.0.6), 30 hops max, 40 byte packets 1 eth-3.gw-4.hck.idt.net (169.132.32.254) 2.907 ms 2.026 ms 3.068 ms 2 eth-0.gw-1.hck.idt.net (169.132.34.250) 107.054 ms 121.335 ms 77.012 ms 3 10.255.255.1 (10.255.255.1) 82.342 ms 70.764 ms * 4 192.41.177.181 (192.41.177.181) 55.270 ms 48.858 ms 42.972 ms 5 core2-hssi2-0.Washington.mci.net (204.70.1.213) 40.422 ms 40.915 ms 43.822 ms 6 border7-fddi-0.Washington.mci.net (204.70.74.51) 33.906 ms 78.689 ms 75.278 ms 7 * * * 8 * * * 9 * * * 10 * * * 11 * * * 12 * * * 13 * * * 14 * * *
------- End of Forwarded Message
-- --bill
On Tue, 20 Aug 1996 bmanning@ISI.EDU wrote:
Network Solutions has decided to filter our CIDR addr, 169.132 / 16 since about 5 PM yesterday from reaching the internic. Below is a forwarded copy of email exchange between myself, and MCI --
It was my understanding that Network Solutions was not a provider, but a customer of several providers, including MCI. Your statements and the attached note from MCI seem to indicate otherwise. Or did this particular outage have anything to do with the MCI instabilities around KC yesterday? Or perhaps with your use of net10 in the path?
It doesn't appear to have anything to do with MCI's instabilities, or or net10 path -- This happens only from our 169.132 addresses, below is a sample traceroute originating from 207.113.31.130 : traceroute to rs.internic.net (198.41.0.6), 30 hops max, 40 byte packets 1 eth-0.gw-1.hh.idt.net (207.113.31.158) 1.717 ms 1.679 ms 1.706 ms 2 lp-6-5.gw-6.hck.idt.net (206.20.143.73) 6.731 ms 3.593 ms 3.588 ms 3 eth-0.gw-1.hck.idt.net (169.132.34.250) 52.896 ms 62.393 ms 14.151 ms 4 bb.gw-1.dc.idt.net (10.255.255.1) 15.797 ms 100.077 ms 94.122 ms 5 mae-east-plusplus.washington.mci.net (192.41.177.181) 289.252 ms 116.522 ms 191.672 ms 6 * core2-hssi2-0.Washington.mci.net (204.70.1.213) 33.655 ms 83.444 ms 7 border7-fddi-0.Washington.mci.net (204.70.74.51) 86.084 ms 98.839 ms 95.518 ms 8 network-solutions.Washington.mci.net (204.70.77.42) 176.883 ms 131.873 ms 162.567 ms 9 rs1.internic.net (198.41.0.6) 109.422 ms * 134.412 ms - Golan
Sorry - we are not filtering. This is a routing problem that I'm sure Alternet will fix (there is a ticket open with the Alternet NOC). As I mentioned in private email, we are preferring an Alternet path back to your site and it is dying within Alternet (although MCI is advertising a more specific /24 out of that /16). If you tried to contact me, why did I not have a message in my voice mail when I got to work today? Mark PS. In the future, we have an email address that deals with this type of thing - try action@internic.net.
On Tue, 20 Aug 1996 bmanning@ISI.EDU wrote:
Network Solutions has decided to filter our CIDR addr, 169.132 / 16 since about 5 PM yesterday from reaching the internic. Below is a forwarded copy of email exchange between myself, and MCI --
-- Mark Kosters markk@internic.net +1 703 742 4795 Principal Investigator InterNIC Registration Services PGP Key fingerprint = 1A 2A 92 F8 8E D3 47 F9 15 65 80 87 68 13 F6 48
Sorry - we are not filtering. This is a routing problem that I'm sure Alternet will fix (there is a ticket open with the Alternet NOC). As I mentioned in private email, we are preferring an Alternet path back to your site and it is dying within Alternet (although MCI is advertising a more specific /24 out of that /16).
If you tried to contact me, why did I not have a message in my voice mail when I got to work today?
MCI and I attempted to get in contact everyone possible yesterday with their contact list by email, including Allwyn Crichlow by voice mail. (They did all the mailing, since our mail would have been deferred.) Date: Mon, 19 Aug 1996 17:11:32 -0400 From: Nour Elouali <nour@mci.net> To: markk@netsole.com, allwync@rs.internic.net Cc: bnite@tremere.ios.com, trouble@mci.net Subject: Access Denied vrfy -e markk@netsol.com Mark Kosters <markk@slam.internic.net> Is this not your email address? Since your information was the missing link, it would have been very helpful to have been in contact yesterday, at 5PM when MCI was doing initial {mis}diag of the situation. The feedback I got from MCI during the last outage was the same feedback I got from them during this outage - Claiming there was a problem with access-lists - The same message passed to you, and Kim.
PS. In the future, we have an email address that deals with this type of thing - try action@internic.net.
If action@internic.net is a better method of contacting you when email is working, what is a better method of getting in contact when email is not working? Considering the importance of the internic's position, is there a 24hr Pager Number? - Golan
Since your information was the missing link, it would have been very helpful to have been in contact yesterday...<SNIP!>
Oh, for crying out loud! "Yes, I'd like to get in on this senseless laundry airing, please." All those with the letter H for a middle initial: knock it off, you're confusing my spell checker! Kim Hubbard really pisses me off because she knows people who wear the color green, and that violates the Freedom of Refrigerators act of 1743 All this business of discriminating against crustaceans must stop NOW! I'm tired of hearing crabs say, "We been dissed, man, we been dissed!" and then the liquor store across town gets raided by lobsters. Oh, and quit doing that stupid macaroni dance; that's how the elevator got broken last week!!! This list isn't the place for this kind of childish banter. For one thing, I expect Kim and Mark and Anyoneelse will respond a lot more favorably if you don't take them out in public and say, "See there! Their skin: it's _pliable!_" and make a monkey out of yourself. Furthermore, I'd be remiss in my duties as Chief Jerk Hate Target if I didn't slam someone this week for over-CCing. Apart from that, one thing you MUST learn is that the difference between accusation and whining is PROOF. When dealing with the InterNIC, especially, proof will win your case faster than a bawl session. Now, be quiet and go sit in the corner. Carl
On Tue, 20 Aug 1996, Golan Ben-Oni wrote:
To: markk@netsole.com, allwync@rs.internic.net
vrfy -e markk@netsol.com Mark Kosters <markk@slam.internic.net>
Is this not your email address?
Am I the only one who notices these things? See that guy over there pointing at you and shaking his head? That's Perry Metzger and he's not sure whether to be pleased that you are demonstrating the accuracy of his cowboy theory or whether to cry.... Michael Dillon - ISP & Internet Consulting Memra Software Inc. - Fax: +1-604-546-3049 http://www.memra.com - E-mail: michael@memra.com
Network Solutions has decided to filter our CIDR addr, 169.132 / 16 since about 5 PM yesterday from reaching the internic. Below is a forwarded copy of email exchange between myself, and MCI --
It appears that there is no backup routing engineer at the Internic which troubles me greatly. MCI has been attempting to contact them every 30 minutes since yesterday to no avail.
This is the second time in a month or so that this has happened.
Since I was unable to contact Mark Kosters, I felt it might be helpful to contact Kim Hubbard - To my dismay, she refused to acknowledge our problem and hung up on me with an attitude. <sigh>
For the record, I did not refuse to acknowledge your problem. I refused to acknowledge your wild accusations that Network Solutions was purposely singling out your network number for filtering. You refused to acknowledge that there could be any other possible reason for your problem. Also, I did not hang up on you, with or without an attitude. Kim Hubbard InterNIC Registry
Of course, this pleases me to no end.
---------- Forwarded message ---------- Date: Mon, 19 Aug 1996 17:11:32 -0400 From: Nour Elouali <nour@mci.net> To: markk@netsole.com, allwync@rs.internic.net Cc: bnite@tremere.ios.com, trouble@mci.net Subject: Access Denied
Hi there,
One of our peers _idt.net_ cannot reach your site, see the forwarded traceroute.. They cannot route to you from any IP Address within their CIDR Block 169.132.0.0/16.
According to my traceroute, I see that they get filtered out at network-solutions which is your provider. Will you be able to assist? Thanks...
-Nour Elouali MCI NOC
******
border7.wtn#tr 198.41.0.6
Type escape sequence to abort. Tracing the route to rs1.internic.net (198.41.0.6)
1 network-solutions.Washington.mci.net (204.70.77.42) 8 msec 8 msec 4 msec 2 rs1.internic.net (198.41.0.6) [AS 6245] 8 msec 4 msec 4 msec ******
------- Forwarded Message
Delivery-Date: Mon, 19 Aug 1996 17:27:36 -0500 Received: from postoffice.Reston.mci.net by carbon.cary.mci.net (8.6.12/kaw-mci.net/feb95) id NAA02326; Mon, 19 Aug 1996 13:27:24 -0400 Received: from tremere.ios.com (tremere.ios.com [169.132.32.250]) by postoffice.Reston.mci.net (8.7.5/8.7.3) with ESMTP id NAA28560 for <trouble@mci.net>; Mon, 19 Aug 1996 13:27:21 -0400 (EDT) Received: (from bnite@localhost) by tremere.ios.com (8.7.5/8.6.9) id NAA08739 for trouble@mci.net; Mon, 19 Aug 1996 13:27:11 -0400 (EDT) Date: Mon, 19 Aug 1996 13:27:11 -0400 (EDT) From: Golan Ben-Oni <bnite@tremere.ios.com> Message-Id: <199608191727.NAA08739@tremere.ios.com> To: trouble@mci.net Subject: Traceroute to Internic
traceroute rs.internic.net traceroute to rs.internic.net (198.41.0.6), 30 hops max, 40 byte packets 1 eth-3.gw-4.hck.idt.net (169.132.32.254) 2.907 ms 2.026 ms 3.068 ms 2 eth-0.gw-1.hck.idt.net (169.132.34.250) 107.054 ms 121.335 ms 77.012 ms 3 10.255.255.1 (10.255.255.1) 82.342 ms 70.764 ms * 4 192.41.177.181 (192.41.177.181) 55.270 ms 48.858 ms 42.972 ms 5 core2-hssi2-0.Washington.mci.net (204.70.1.213) 40.422 ms 40.915 ms 43.822 ms 6 border7-fddi-0.Washington.mci.net (204.70.74.51) 33.906 ms 78.689 ms 75.278 ms 7 * * * 8 * * * 9 * * * 10 * * * 11 * * * 12 * * * 13 * * * 14 * * *
------- End of Forwarded Message
On Tue, 20 Aug 1996, Kim Hubbard wrote:
For the record, I did not refuse to acknowledge your problem. I refused to acknowledge your wild accusations that Network Solutions was purposely singling out your network number for filtering. You refused to acknowledge that there could be any other possible reason for your problem.
You forgot to point out that his evidence below indicates that this is an MCI problem in Washington. Besides the fact that Network Solutions isn't anyone's provider...
According to my traceroute, I see that they get filtered out at network-solutions which is your provider. Will you be able to assist? Thanks...
traceroute to rs.internic.net (198.41.0.6), 30 hops max, 40 byte packets 1 eth-3.gw-4.hck.idt.net (169.132.32.254) 2.907 ms 2.026 ms 3.068 ms 2 eth-0.gw-1.hck.idt.net (169.132.34.250) 107.054 ms 121.335 ms 77.012 ms 3 10.255.255.1 (10.255.255.1) 82.342 ms 70.764 ms * 4 192.41.177.181 (192.41.177.181) 55.270 ms 48.858 ms 42.972 ms 5 core2-hssi2-0.Washington.mci.net (204.70.1.213) 40.422 ms 40.915 ms 43.822 ms 6 border7-fddi-0.Washington.mci.net (204.70.74.51) 33.906 ms 78.689 ms 75.278 ms 7 * * *
Michael Dillon - ISP & Internet Consulting Memra Software Inc. - Fax: +1-604-546-3049 http://www.memra.com - E-mail: michael@memra.com
Date: Tue, 20 Aug 1996 18:04:03 -0700 (PDT) From: Michael Dillon <michael@memra.com> To: nanog@merit.edu
You forgot to point out that his evidence below indicates that this is an MCI problem in Washington. Besides the fact that Network Solutions isn't anyone's provider...
Let us be more careful here. The traceroute only indicates that the problem was with the hop after border7.wtn.mci.net. The traceroute simply does not contain sufficient information to pinpoint which party the next hop is. In fact, the next hop is the Internic router (204.70.77.42): [foghorn] enke% traceroute 198.41.0.6 traceroute to 198.41.0.6 (198.41.0.6), 30 hops max, 40 byte packets 1 cpe1-ethernet1-1 (204.70.133.1) 4 ms 2 ms 2 ms 2 borderx1-fddi-2.Reston.mci.net (204.70.176.101) 2 ms 2 ms 2 ms 3 core1-hssi-3.Greensboro.mci.net (204.70.1.161) 11 ms 11 ms 10 ms 4 core2-aip-4.Greensboro.mci.net (204.70.1.22) 11 ms 11 ms 11 ms 5 core2-hssi-3.Washington.mci.net (204.70.1.130) 19 ms 19 ms 19 ms 6 border7-fddi-0.Washington.mci.net (204.70.74.51) 19 ms 19 ms 29 ms 7 network-solutions.Washington.mci.net (204.70.77.42) 22 ms 140 ms 209 ms 8 rs1.internic.net (198.41.0.6) 21 ms 21 ms 21 ms As Mark Koster has pointed out, it was a routing problem with the return traffic from Internic and it was not related to MCI. Hope this helps in clarifying the issue. -- Enke
According to my traceroute, I see that they get filtered out at network-solutions which is your provider. Will you be able to assist? Thanks...
traceroute to rs.internic.net (198.41.0.6), 30 hops max, 40 byte packets 1 eth-3.gw-4.hck.idt.net (169.132.32.254) 2.907 ms 2.026 ms 3.068 ms 2 eth-0.gw-1.hck.idt.net (169.132.34.250) 107.054 ms 121.335 ms 77.0
12 ms
3 10.255.255.1 (10.255.255.1) 82.342 ms 70.764 ms * 4 192.41.177.181 (192.41.177.181) 55.270 ms 48.858 ms 42.972 ms 5 core2-hssi2-0.Washington.mci.net (204.70.1.213) 40.422 ms 40.915 ms
43.822 ms 6 border7-fddi-0.Washington.mci.net (204.70.74.51) 33.906 ms 78.689 m s 75.278 ms 7 * * *
Michael Dillon - ISP & Internet Consulting Memra Software Inc. - Fax: +1-604-546-3049 http://www.memra.com - E-mail: michael@memra.com
On Wed, 21 Aug 1996, Enke Chen wrote:
You forgot to point out that his evidence below indicates that this is an MCI problem in Washington. Besides the fact that Network Solutions isn't anyone's provider...
Let us be more careful here. The traceroute only indicates that the problem was with the hop after border7.wtn.mci.net. The traceroute simply does not contain sufficient information to pinpoint which party the next hop is.
Very true. That fact and the widespread use of assymetric return paths (as in this case) really makes it impossible to authoritatively diagnose who is at fault when a routing problem occurs merely by reading a few traceroutes. One thing that would help diagnose the true source of the problem with less human interaction required (avoiding voicemail and all that) would be wider public access to traceroutes beginning at key sites. There is a group of ISP's who make CGI-based traceroutes available from their sites at http://amazing.netaxs.com/internet/club-traceroute.html In this instance, it would have helped if the Internic had a CGI traceroute available from their site but I think it would also be a nice thing if the major NSP's would make this same kind of facility available from their exchange points. What do you folks think? Michael Dillon - ISP & Internet Consulting Memra Software Inc. - Fax: +1-604-546-3049 http://www.memra.com - E-mail: michael@memra.com
In a previous message, Michael Dillon wrote:
[snip]
One thing that would help diagnose the true source of the problem with less human interaction required (avoiding voicemail and all that) would be wider public access to traceroutes beginning at key sites. There is a group of ISP's who make CGI-based traceroutes available from their sites at http://amazing.netaxs.com/internet/club-traceroute.html
In this instance, it would have helped if the Internic had a CGI traceroute available from their site but I think it would also be a nice thing if the major NSP's would make this same kind of facility available from their exchange points.
FWIW, Digex has such an animal which appears to run commands on their routers at MAE-E, MAE-W, and the Sprint NAP: <http://nitrous.digex.net/> -- David Carmean WB6YZM DC574 <dlc@silcom.com> System/Network Administration, Silicon Beach Communications Unsolicited commercial e-mail not accepted. Violators will be LARTed.
As Michael Dillon once put it so eloquently:
One thing that would help diagnose the true source of the problem with less human interaction required (avoiding voicemail and all that) would be wider public access to traceroutes beginning at key sites. There is a group of ISP's who make CGI-based traceroutes available from their sites at http://amazing.netaxs.com/internet/club-traceroute.html
Another WWW list of publicly accessible traceroutes is at http://www.freenix.fr This page is in French, but you should be able to pick out the words "traceroute" and "outils reseau" (network utilities). Both of these pages are in English. (This site also has links to various free Unices.) Christopher A. Bongaarts /\ Email: Chris.Bongaarts-2@umn.edu Integrated Systems Solutions /\ Office: 130A Lind Hall Office of Information Technology /\ Phone: +1 (612) 625-1809 University of Minnesota /\ WWW: http://umn.edu/~cab
perhaps if folks would just take the time to loose source route in both directions instead of only one... was the web page in response to complaints to those who turn off lsrr? Jeff Young young@mci.net
To: nanog@merit.edu Subject: Re: Access to the Internic Blocked
On Wed, 21 Aug 1996, Enke Chen wrote:
You forgot to point out that his evidence below indicates that this is an MCI problem in Washington. Besides the fact that Network Solutions isn't anyone's provider...
Let us be more careful here. The traceroute only indicates that the problem was with the hop after border7.wtn.mci.net. The traceroute simply does not contain sufficient information to pinpoint which party the next hop is.
Very true. That fact and the widespread use of assymetric return paths (as in this case) really makes it impossible to authoritatively diagnose who is at fault when a routing problem occurs merely by reading a few traceroutes.
One thing that would help diagnose the true source of the problem with less human interaction required (avoiding voicemail and all that) would be wider public access to traceroutes beginning at key sites. There is a group of ISP's who make CGI-based traceroutes available from their sites at http://amazing.netaxs.com/internet/club-traceroute.html
In this instance, it would have helped if the Internic had a CGI traceroute available from their site but I think it would also be a nice thing if the major NSP's would make this same kind of facility available from their exchange points.
What do you folks think?
Michael Dillon - ISP & Internet Consulting Memra Software Inc. - Fax: +1-604-546-3049 http://www.memra.com - E-mail: michael@memra.com
participants (10)
-
bmanning＠ISI.EDU -
Carl Payne -
Chris Bongaarts -
dlc＠silcom.com -
Enke Chen -
Golan Ben-Oni -
Jeff Young -
Kim Hubbard -
Mark Kosters -
Michael Dillon