Dear NANOG-ers, I work for an information security company that is dependant upon ICMP for network mapping purposes (read: traceroute). On or about August 18, we were told, our upstream provider began blocking ICMP packets at its border in the Chicago NAP in an effort to cut down on the propagation of 'MSBlast'. This has effected our ability to accurately map our customers networks. We've been in contact with an engineer in this provider's NOC who is either unable or unwilling to remove this ACL for our block of IPs. Currently, we've been given two options. (1) Deal with the effect of the ACL until 'MSBlast' traffic subsides, or (2) they are willing to reroute our traffic out of the Chicago NAP to a border router that, they claim, does not have the same ACL. The problem with option 2 is that they would force us to renumber. This is a problem for us, as it would impact our customers as well. What options can I take to my management that would cause the least impact to the services we provide while not causing undue work for our clients. Also, what other options could I suggest to my upstream provider? TIA, C. Windon __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com
CA Windon wrote:
Dear NANOG-ers,
I work for an information security company that is dependant upon ICMP for network mapping purposes (read: traceroute). On or about August 18, we were told, our upstream provider began blocking ICMP packets at its border in the Chicago NAP in an effort to cut down on the propagation of 'MSBlast'. This has effected our ability to accurately map our customers networks.
We've been in contact with an engineer in this provider's NOC who is either unable or unwilling to remove this ACL for our block of IPs.
Currently, we've been given two options. (1) Deal with the effect of the ACL until 'MSBlast' traffic subsides, or (2) they are willing to reroute our traffic out of the Chicago NAP to a border router that, they claim, does not have the same ACL. The problem with option 2 is that they would force us to renumber. This is a problem for us, as it would impact our customers as well.
What options can I take to my management that would cause the least impact to the services we provide while not causing undue work for our clients. Also, what other options could I suggest to my upstream provider?
Blocking ICMP in no way slows or prevents the propagation of MSBlaster. ICMP echo requests and responses are, however, a byproduct of the Welchia/Nachi worm and blocking this traffic will prevent the worm's spread. Tell your ISP it need _at most_ block ICMP echoes. If they are blocking ICMP unreachables, which would break your traceroutes, they have broken the Internet Protocol. (Period.) One can even be more specific about blocking ICMP echo requests of a certain, atypical size to stop the Welchia pings while letting other ICMP pass. See the list archives for detailed instruction for how to do this for a variety of router platforms. -- Crist J. Clark crist.clark@globalstar.com Globalstar Communications (408) 933-4387 The information contained in this e-mail message is confidential, intended only for the use of the individual or entity named above. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any review, dissemination, distribution or copying of this communication is strictly prohibited. If you have received this e-mail in error, please contact postmaster@globalstar.com
<rant> Providers blocking all ICMP = ignorant I can't possibly stand any ISP's blocking _ALL_ ICMP (alas it is happening now, I already know 5 ISP's around my area who's doing this as I speak) for any reasons. If you want to *cough*cough*mitigate*/cough*/cough* impact of so-called BLASTER, please please please for the love of god, just block echo/echo replies. Not to mention blocking icmp will not help stop the propagation of the worm. </rant> -hc -- Haesu C. TowardEX Technologies, Inc. Consulting, colocation, web hosting, network design and implementation http://www.towardex.com | haesu@towardex.com Cell: (978)394-2867 | Office: (978)263-3399 Ext. 174 Fax: (978)263-0033 | POC: HAESU-ARIN On Mon, Sep 29, 2003 at 09:43:14AM -0700, CA Windon wrote:
Dear NANOG-ers,
I work for an information security company that is dependant upon ICMP for network mapping purposes (read: traceroute). On or about August 18, we were told, our upstream provider began blocking ICMP packets at its border in the Chicago NAP in an effort to cut down on the propagation of 'MSBlast'. This has effected our ability to accurately map our customers networks.
We've been in contact with an engineer in this provider's NOC who is either unable or unwilling to remove this ACL for our block of IPs.
Currently, we've been given two options. (1) Deal with the effect of the ACL until 'MSBlast' traffic subsides, or (2) they are willing to reroute our traffic out of the Chicago NAP to a border router that, they claim, does not have the same ACL. The problem with option 2 is that they would force us to renumber. This is a problem for us, as it would impact our customers as well.
What options can I take to my management that would cause the least impact to the services we provide while not causing undue work for our clients. Also, what other options could I suggest to my upstream provider?
TIA,
C. Windon
__________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com
dependant upon ICMP for network mapping purposes (read: traceroute). On or about August 18, we were told, our upstream provider began blocking ICMP
If your uptream is only blocking 92 bute ICMP.. other programs such as WinMTR http://www.icnet.ro/~stanimir/winmtr/ should work. If they are blocking ALL ICMP traffic, well.. a whole slew of issues is at hand.
Hmm noticed what I was to say has already been said, but to reiterate, if your provider is blocking ICMP other than echo/echoreply .. in this case ICMP unreachables and presumably fragments and other fundementally required icmps they are seriously broken and I would insist they fix it or else you move away You didnt clarify that in your mail tho, is it the icmp unreachables that you arent getting or is your monitoring sending out icmp echos which are being filtering? if its the latter then you can easily workaround by modifying your monitoring systems to use udp/tcp based probes which are probably better these days than sending icmp across third party networks anyhow Steve On Mon, 29 Sep 2003, CA Windon wrote:
Dear NANOG-ers,
I work for an information security company that is dependant upon ICMP for network mapping purposes (read: traceroute). On or about August 18, we were told, our upstream provider began blocking ICMP packets at its border in the Chicago NAP in an effort to cut down on the propagation of 'MSBlast'. This has effected our ability to accurately map our customers networks.
We've been in contact with an engineer in this provider's NOC who is either unable or unwilling to remove this ACL for our block of IPs.
Currently, we've been given two options. (1) Deal with the effect of the ACL until 'MSBlast' traffic subsides, or (2) they are willing to reroute our traffic out of the Chicago NAP to a border router that, they claim, does not have the same ACL. The problem with option 2 is that they would force us to renumber. This is a problem for us, as it would impact our customers as well.
What options can I take to my management that would cause the least impact to the services we provide while not causing undue work for our clients. Also, what other options could I suggest to my upstream provider?
TIA,
C. Windon
__________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com
winders does use udp instead of icmp in their tracert program, IIRC (or at least they used to). At the risk of getting my head blown off, could we say that was foresight :) Eric
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Stephen J. Wilcox Sent: Monday, September 29, 2003 1:54 PM To: CA Windon Cc: nanog@merit.edu Subject: Re: ICMP Blocking Woes
Hmm noticed what I was to say has already been said, but to reiterate, if your provider is blocking ICMP other than echo/echoreply .. in this case ICMP unreachables and presumably fragments and other fundementally required icmps they are seriously broken and I would insist they fix it or else you move away
You didnt clarify that in your mail tho, is it the icmp unreachables that you arent getting or is your monitoring sending out icmp echos which are being filtering?
if its the latter then you can easily workaround by modifying your monitoring systems to use udp/tcp based probes which are probably better these days than sending icmp across third party networks anyhow
Steve
On Mon, 29 Sep 2003, CA Windon wrote:
Dear NANOG-ers,
I work for an information security company that is dependant upon ICMP for network mapping purposes (read: traceroute). On or about August 18, we were told, our upstream provider began blocking ICMP packets at its border in the Chicago NAP in an effort to cut down on the propagation of 'MSBlast'. This has effected our ability to accurately map our customers networks.
We've been in contact with an engineer in this provider's NOC who is either unable or unwilling to remove this ACL for our block of IPs.
Currently, we've been given two options. (1) Deal with the effect of the ACL until 'MSBlast' traffic subsides, or (2) they are willing to reroute our traffic out of the Chicago NAP to a border router that, they claim, does not have the same ACL. The problem with option 2 is that they would force us to renumber. This is a problem for us, as it would impact our customers as well.
What options can I take to my management that would cause the least impact to the services we provide while not causing undue work for our clients. Also, what other options could I suggest to my upstream provider?
TIA,
C. Windon
__________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com
In message <NDBBJJPLIGJGLBKILFIHMEPLMIAA.ekgermann@cctec.com>, "Eric Germann" w rites:
winders does use udp instead of icmp in their tracert program, IIRC (or at least they used to). At the risk of getting my head blown off, could we say that was foresight :)
No, they use icmp. Or at least that's what the XP box sitting next to me does... --Steve Bellovin, http://www.research.att.com/~smb
On Mon, 2003-09-29 at 16:10, Steven M. Bellovin wrote:
In message <NDBBJJPLIGJGLBKILFIHMEPLMIAA.ekgermann@cctec.com>, "Eric Germann" w rites:
winders does use udp instead of icmp in their tracert program, IIRC (or at least they used to). At the risk of getting my head blown off, could we say that was foresight :)
No, they use icmp. Or at least that's what the XP box sitting next to me does...
So far I've seen is it uses UDP with a TTL that increments by one for each hop. The ICMP time exceeded message is returned from the interface of the router closest to you, and then windows tries to ping the hop. If it can't do this, it displays * * *. Why it needs do this rather than simply use only UDP like the rest of the world, I don't know. But leave it to microsoft to be different... -Paul -- Paul Timmins <paul@timmins.net>
SMB> Date: Mon, 29 Sep 2003 16:10:59 -0400 SMB> From: Steven M. Bellovin SMB> No, they use icmp. Or at least that's what the XP box SMB> sitting next to me does... AFAIK, it's been that way since Win95. I recall a certain vendor's dodgy ISDN router * * * on Windows traceroute, but working fine under *ix... for whatever reason, said router didn't like the ICMP traceroute, but returned unreachables in response to UDP when TTL expired. Eddy -- Brotsman & Dreger, Inc. - EverQuick Internet Division Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita _________________________________________________________________ DO NOT send mail to the following addresses : blacklist@brics.com -or- alfra@intc.net -or- curbjmp@intc.net Sending mail to spambait addresses is a great way to get blocked.
AFAIK, it's been that way since Win95. I recall a certain vendor's dodgy ISDN router * * * on Windows traceroute, but working fine under *ix... for whatever reason, said router didn't like the ICMP traceroute, but returned unreachables in response to UDP when TTL expired.
WindowsNT tracert.exe uses 92 byte icmp packets. There is a modified version that uses a smaller sized icmp packet at http://www.nthelp.com/NT6/tracert_broken.htm that works fine on Windows 2000. Geo.
WindowsNT tracert.exe uses 92 byte icmp packets. There is a modified version that uses a smaller sized icmp packet at http://www.nthelp.com/NT6/tracert_broken.htm that works fine on Windows 2000.
So if tracert1 doesn't work, would that mean Comcast is actually blocking all ICMP ? I have been told they are only blocking 135-139, 4444 I get the same results with tracert and tracert1 (below) D:\Temp>ver Microsoft Windows 2000 [Version 5.00.2195] D:\Temp>tracert1 www.advil.com Tracing route to www.advil.com [164.109.5.98] over a maximum of 30 hops: 1 20 ms 10 ms 10 ms c-24-218-168-1.ne.client2.attbi.com [24.218.168.1] 2 20 ms 10 ms 10 ms 24.62.0.245 3 * * * Request timed out. 4 * * * Request timed out. 5 * * * Request timed out. 6 * * * Request timed out. 7 * * * Request timed out. D:\Temp>tracert www.advil.com Tracing route to www.advil.com [164.109.5.98] over a maximum of 30 hops: 1 20 ms 10 ms 10 ms c-24-218-168-1.ne.client2.attbi.com [24.218.168.1] 2 10 ms 10 ms 20 ms 24.62.0.245 3 * * * Request timed out. 4 * * * Request timed out. 5 * * * Request timed out. 6 * ^C Eric
WindowsNT tracert.exe uses 92 byte icmp packets. There is a modified version that uses a smaller sized icmp packet at http://www.nthelp.com/NT6/tracert_broken.htm that works fine on Windows 2000.
So if tracert1 doesn't work, would that mean Comcast is actually blocking all ICMP ? I have been told they are only blocking 135-139, 4444 I get the same results with tracert and tracert1 (below) D:\Temp>ver Microsoft Windows 2000 [Version 5.00.2195] D:\Temp>tracert1 www.advil.com Tracing route to www.advil.com [164.109.5.98] over a maximum of 30 hops: 1 20 ms 10 ms 10 ms c-24-218-168-1.ne.client2.attbi.com [24.218.168.1] 2 20 ms 10 ms 10 ms 24.62.0.245 3 * * * Request timed out. 4 * * * Request timed out. 5 * * * Request timed out. 6 * * * Request timed out. 7 * * * Request timed out. D:\Temp>tracert www.advil.com Tracing route to www.advil.com [164.109.5.98] over a maximum of 30 hops: 1 20 ms 10 ms 10 ms c-24-218-168-1.ne.client2.attbi.com [24.218.168.1] 2 10 ms 10 ms 20 ms 24.62.0.245 3 * * * Request timed out. 4 * * * Request timed out. 5 * * * Request timed out. 6 * ^C Eric
They are filtering either ICMP echo or echo reply; using an LBNL/Unix traceroute is successful the entire path. Eric Kagan wrote:
WindowsNT tracert.exe uses 92 byte icmp packets. There is a modified
version
that uses a smaller sized icmp packet at http://www.nthelp.com/NT6/tracert_broken.htm that works fine on Windows 2000.
So if tracert1 doesn't work, would that mean Comcast is actually blocking all ICMP ? I have been told they are only blocking 135-139, 4444 I get the same results with tracert and tracert1 (below)
D:\Temp>ver
Microsoft Windows 2000 [Version 5.00.2195]
D:\Temp>tracert1 www.advil.com
Tracing route to www.advil.com [164.109.5.98] over a maximum of 30 hops:
1 20 ms 10 ms 10 ms c-24-218-168-1.ne.client2.attbi.com [24.218.168.1] 2 20 ms 10 ms 10 ms 24.62.0.245 3 * * * Request timed out. 4 * * * Request timed out. 5 * * * Request timed out. 6 * * * Request timed out. 7 * * * Request timed out.
D:\Temp>tracert www.advil.com
Tracing route to www.advil.com [164.109.5.98] over a maximum of 30 hops:
1 20 ms 10 ms 10 ms c-24-218-168-1.ne.client2.attbi.com [24.218.168.1] 2 10 ms 10 ms 20 ms 24.62.0.245 3 * * * Request timed out. 4 * * * Request timed out. 5 * * * Request timed out. 6 * ^C
Eric
AFAIK, it's been that way since Win95. I recall a certain vendor's dodgy ISDN router * * * on Windows traceroute, but working fine under *ix... for whatever reason, said router didn't like the ICMP traceroute, but returned unreachables in response to UDP when TTL expired.
Eddy
Wasn't this based upon the premise that gear should not return ICMP errors as a result of ICMP packet input as a precaution against error loops? ie said dodgy router did the _right_ thing?
bdragon@gweep.net wrote:
AFAIK, it's been that way since Win95. I recall a certain vendor's dodgy ISDN router * * * on Windows traceroute, but working fine under *ix... for whatever reason, said router didn't like the ICMP traceroute, but returned unreachables in response to UDP when TTL expired.
Eddy
Wasn't this based upon the premise that gear should not return ICMP errors as a result of ICMP packet input as a precaution against error loops? ie said dodgy router did the _right_ thing?
That would be disingenious. RFC1122 clearly lists which ICMP are error messages, 3.2.2 Internet Control Message Protocol -- ICMP ICMP messages are grouped into two classes. * ICMP error messages: Destination Unreachable (see Section 3.2.2.1) Redirect (see Section 3.2.2.2) Source Quench (see Section 3.2.2.3) Time Exceeded (see Section 3.2.2.4) Parameter Problem (see Section 3.2.2.5) * ICMP query messages: Echo (see Section 3.2.2.6) Information (see Section 3.2.2.7) Timestamp (see Section 3.2.2.8) Address Mask (see Section 3.2.2.9) But it would not surprise me one bit if some lazy coder actually didn't do what you describe just to make the code simpler and try to use that as a justification. -- Crist J. Clark crist.clark@globalstar.com Globalstar Communications (408) 933-4387
On Tue, Sep 30, 2003 at 05:22:25PM -0700, Crist Clark wrote:
Wasn't this based upon the premise that gear should not return ICMP errors as a result of ICMP packet input as a precaution against error loops? ie said dodgy router did the _right_ thing?
That would be disingenious. RFC1122 clearly lists which ICMP are error messages,
The following from W. Richard Stevens' archive presents some additional insight: <http://www.kohala.com/start/papers.others/vanj.99feb08.txt> John
John Kristoff wrote:
On Tue, Sep 30, 2003 at 05:22:25PM -0700, Crist Clark wrote:
Wasn't this based upon the premise that gear should not return ICMP errors as a result of ICMP packet input as a precaution against error loops? ie said dodgy router did the _right_ thing?
That would be disingenious. RFC1122 clearly lists which ICMP are error messages,
The following from W. Richard Stevens' archive presents some additional insight:
<http://www.kohala.com/start/papers.others/vanj.99feb08.txt>
But if you take that quote from RFC792 absolutely literally, ...no ICMP messages are sent about ICMP messages. You shouldn't ever respond to a echo request with an echo reply, or timestamp requests/responses, or netmask request/responses, etc. -- Crist J. Clark crist.clark@globalstar.com Globalstar Communications (408) 933-4387 The information contained in this e-mail message is confidential, intended only for the use of the individual or entity named above. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any review, dissemination, distribution or copying of this communication is strictly prohibited. If you have received this e-mail in error, please contact postmaster@globalstar.com
My thanks to everyone for their responses. The consensus seems to be to get a new provider, epecially if my current one is willing to break Internet Protocol. To clarify, since the question was posed several times, we were told that they are blocking _all_ ICMP. C. Windon __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com
Various ISPs have been trying lots of different ICMP filters. You can see some of the impact on the Internet average graphs from XAffire. http://www.xaffire.com/press/ea/EA20030902_images?rf=EM005 Xaffire/Matrix Systems apparently used ping packets that were the same size as those being filtered by some ISPs. According to Xaffire service providers implementing filters included Cable & Wireless and Level 3.
Lo! On Thu, 2 Oct 2003, Sean Donelan did sayeth:
Various ISPs have been trying lots of different ICMP filters. You can see some of the impact on the Internet average graphs from XAffire.
http://www.xaffire.com/press/ea/EA20030902_images?rf=EM005
Xaffire/Matrix Systems apparently used ping packets that were the same size as those being filtered by some ISPs. According to Xaffire service providers implementing filters included Cable & Wireless and Level 3.
It does raise the question of whether ICMP Echo is a good mechanism for monitoring systems that are across third party networks. I personally think that filtering ICMP is becoming less useful and you would get better results using other probe methods eg SYN/RST as deployed by numerous port scanning tools eg nmap Steve
On Thu, 2 Oct 2003, Stephen J. Wilcox wrote:
It does raise the question of whether ICMP Echo is a good mechanism for monitoring systems that are across third party networks.
I personally think that filtering ICMP is becoming less useful and you would get better results using other probe methods eg SYN/RST as deployed by numerous port scanning tools eg nmap
The problems of PING monitoring have been around for a long time. SRI-NIC.ARPA had to block PING in 1987 because so many sites kept pinging the NIC, it was causing problems. I recall, but can't find, in the old ARPANET a memo about the problem of people pinging the IMP gateways. The advantage of using PING is the site can block or rate-limit PING without effecting their "real" services. Using SYN/RST is a higher overhead probe, leaving the host with fewer alternatives when the "monitoring" packets causes problems with the other services. Most high visibility sites, like the Root Servers, Yahoo, Google, CNN, BBC, Whitehouse.Gov, etc are under almost constant "attack" from people monitoring their reachability. Almost no third-party monitors ask permission to engage in the constant pinging/monitoring of the sites. The Department of Defense used to report every PING or Traceroute attempt as an "attack" on their networks. It was great for generating huge numbers for Congress when asking for more money, but is it really a usefull measurement. PING is a useful tool. But if the target host blocks ping, it probably shouldn't be considered an invitation to "monitor" the site with more intrusive methods. On the other hand, if ISPs had zero tolorance policies and enforced every term of their AUP in every instance, virtually every network tool and network engineer would be considered network abuse.
participants (15)
-
Anthony Cennami -
bdragon＠gweep.net -
CA Windon -
Crist Clark -
E.B. Dreger -
Eric Germann -
Eric Kagan -
Geo. -
Haesu -
John Kristoff -
mike harrison -
Paul Timmins -
Sean Donelan -
Stephen J. Wilcox -
Steven M. Bellovin