RE: 69/8...this sucks -- Centralizing filtering..
What surprises me most about this entire thread is the lack of centralized filtering. Since most service providers should be thinking about a sink hole network for security auditing (and backscatter), why not have ONE place where you advertise all unreachable, or better yet -- a default (ie everything NOT learned through BGP peers), and just forward the packets to a bit bucket.. Which is better than an access list since, now we are forwarding packets instead of sending them to a CPU to increase router load. I don't think ARIN can help the situation. ISPs just need to remove the access lists from each router in the network and centralize them. Regards, mark -- Mark Segal Director, Data Services Futureway Communications Inc. Tel: (905)326-1570
-----Original Message----- From: E.B. Dreger [mailto:eddy+public+spam@noc.everquick.net] Sent: March 10, 2003 10:17 AM To: nanog@merit.edu Subject: Re: 69/8...this sucks
Date: Mon, 10 Mar 2003 09:46:33 +0000 From: Michael.Dillon
I have suggested that ARIN should set up an LDAP server to publish the delegation of all their IP address space updated
Not bad, but will the lazy ISPs set up an LDAP server to track changes they aren't tracking now? Will those with erroneous filters magically change simply because of LDAP? I still contend the answer is is a boot to the head that screams to them, "Update your freaking filters!"
Eddy -- Brotsman & Dreger, Inc. - EverQuick Internet Division Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 (785) 865-5885 Lawrence and [inter]national Phone: +1 (316) 794-8922 Wichita
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature.
These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
MS> Date: Mon, 10 Mar 2003 10:27:35 -0500 MS> From: Mark Segal MS> Since most service providers should be thinking about a sink MS> hole network for security auditing (and backscatter), why MS> not have ONE place where you advertise all unreachable, or MS> better yet -- a default (ie everything NOT learned through MS> BGP peers), and just forward the packets to a bit bucket.. MS> Which is better than an access list since, now we are MS> forwarding packets instead of sending them to a CPU to MS> increase router load. Chris Morrow and Brian Gemberling (a.k.a. dies) have some fine instructions on how to do just that. Rob Thomas has a bogon route server that comes in handy. The problem with only a default: Think when a rogue ISP decides to advertise an unused netblock and utilize that IP space for malicious purposes. A route exists... do we trust it? MS> I don't think ARIN can help the situation. ISPs just need to Probably not. Nor should they need to. Although perhaps they could allocate other netblocks, and they _do_ charge a fair amount for PI space... ;-) MS> remove the access lists from each router in the network and MS> centralize them. Now, how can we force that? Sufficient reward for doing so, or pain for failure. Evidently "some people can't reach you" isn't enough pain, and having full reachability isn't enough reward. I'm looking forward to Jon Lewis (or others) providing some stats about just how bad the problem is... being fortunate enough not to have [any clients in] 69/8 space I can't comment first-hand on the severity of the problem. Eddy -- Brotsman & Dreger, Inc. - EverQuick Internet Division Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 (785) 865-5885 Lawrence and [inter]national Phone: +1 (316) 794-8922 Wichita ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature. These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
On Mon, 10 Mar 2003, E.B. Dreger wrote:
Now, how can we force that? Sufficient reward for doing so, or pain for failure. Evidently "some people can't reach you" isn't enough pain, and having full reachability isn't enough reward.
I think the only way that's relatively guaranteed to be effective is to move a critical resource (like the gtld-servers) into new IP blocks when previously reserved blocks are assigned to RIR's. I still have a couple hundred thousand IPs to check (I'm going to step up the pace and see if I can get through the list today), but I already have a list of several hundred IPs in networks that ignore 69/8. The list includes such networks as NASA, the US DoD, and networks in China, Russia, and Poland. Those are just a few that I've done manual whois's for. I haven't decided yet whether I'll send automated messages to all the broken networks and give them time to respond and fix their filters, or just post them all to NANOG when the list is complete. Are people interested in seeing the full list (at least the ones I find) of networks that filter 69/8? Does Atlantic.Net get an ARIN discount for doing all this leg work? :) ---------------------------------------------------------------------- Jon Lewis *jlewis@lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
I still have a couple hundred thousand IPs to check (I'm going to step up the pace and see if I can get through the list today), but I already have a list of several hundred IPs in networks that ignore 69/8. The list includes such networks as NASA, the US DoD, and networks in China, Russia, and Poland. Those are just a few that I've done manual whois's for.
You have been busy!
I haven't decided yet whether I'll send automated messages to all the broken networks and give them time to respond and fix their filters, or just post them all to NANOG when the list is complete.
Are people interested in seeing the full list (at least the ones I find) of networks that filter 69/8?
Why not do a weekly report of some sort. Post a summary to nanog with a reference to the website containing the full list. If you can group them by ASN you can do a report a la cidr-report of top 20 offenders and include that in your nanog post. I think this would be a good way to sustain momentum in your worthy cause.. Steve
Does Atlantic.Net get an ARIN discount for doing all this leg work? :)
---------------------------------------------------------------------- Jon Lewis *jlewis@lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
Monday, March 10, 2003, 9:52:06 AM, you wrote: jlo> I think the only way that's relatively guaranteed to be effective is to jlo> move a critical resource (like the gtld-servers) into new IP blocks when jlo> previously reserved blocks are assigned to RIR's. I agree with you. But then since I've been allocated 69/8 I guess you can say I'm a bit biased. jlo> I still have a couple hundred thousand IPs to check (I'm going to step up jlo> the pace and see if I can get through the list today), but I already have jlo> a list of several hundred IPs in networks that ignore 69/8. The list jlo> includes such networks as NASA, the US DoD, and networks in China, Russia, jlo> and Poland. Those are just a few that I've done manual whois's for. jlo> I haven't decided yet whether I'll send automated messages to all the jlo> broken networks and give them time to respond and fix their filters, or jlo> just post them all to NANOG when the list is complete. jlo> Are people interested in seeing the full list (at least the ones I find) jlo> of networks that filter 69/8? Again, since I've been recently allocated in the 69/8 range, I'd love to see this completed list. We've only renumbered our internal workstations into this range, so no customer nets are affected as of yet. But we are about to plunge into our renumbering, so I'm sure customers are going to start yelling then. However, I think this is going to be an on-going problem, even if the gtld-servers were renumbered into 69/8. Do a simple Google search on ip firewalling. You'll find lots of examples using ipchains, iptables, etc, that show example configs. These example configs usually show 69/8 as a bogon network and recommends filtering them. So, in my opinion it's only going to be a matter of time before some network administrator looking to implement a firewall stumbles across one of these broken sample configs and breaks connectivity to me again. In essence, it's going to be an ongoing problem, sure we can fix networks now that we know are broken, but it's going to be an ongoing problem that we are going to have to deal with. Regards, Joe Boyce --- InterStar, Inc. - Shasta.com Internet Phone: +1 (530) 224-6866 x105 Email: jboyce@shasta.com
Since most service providers should be thinking about a sink hole network for security auditing (and backscatter), why not have ONE place where you advertise all unreachable, or better yet -- a default (ie everything NOT learned through BGP peers), and just forward the packets to a bit bucket.. Which is better than an access list since, now we are forwarding packets instead of sending them to a CPU to increase router load.
I don't think ARIN can help the situation. ISPs just need to remove the access lists from each router in the network and centralize them.
I totally agree with you. However, as always, centralized systems, while ease management and scalability, everything becomes a trust issue and a single point of failure or source of problems... May be, this could be a subscription based type of service, something like RADB, where everyone subscribes into a central filtering list that is managed by a seperate organization? I really like the Rob's bogon route-server setup. -hc
Regards, mark
-- Mark Segal Director, Data Services Futureway Communications Inc. Tel: (905)326-1570
-----Original Message----- From: E.B. Dreger [mailto:eddy+public+spam@noc.everquick.net] Sent: March 10, 2003 10:17 AM To: nanog@merit.edu Subject: Re: 69/8...this sucks
Date: Mon, 10 Mar 2003 09:46:33 +0000 From: Michael.Dillon
I have suggested that ARIN should set up an LDAP server to publish the delegation of all their IP address space updated
Not bad, but will the lazy ISPs set up an LDAP server to track changes they aren't tracking now? Will those with erroneous filters magically change simply because of LDAP? I still contend the answer is is a boot to the head that screams to them, "Update your freaking filters!"
Eddy -- Brotsman & Dreger, Inc. - EverQuick Internet Division Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 (785) 865-5885 Lawrence and [inter]national Phone: +1 (316) 794-8922 Wichita
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature.
These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
On Monday, Mar 10, 2003, at 10:54 Canada/Eastern, Haesu wrote:
Since most service providers should be thinking about a sink hole network for security auditing (and backscatter), why not have ONE place where you advertise all unreachable, or better yet -- a default (ie everything NOT learned through BGP peers), and just forward the packets to a bit bucket.. Which is better than an access list since, now we are forwarding packets instead of sending them to a CPU to increase router load.
I don't think ARIN can help the situation. ISPs just need to remove the access lists from each router in the network and centralize them.
I totally agree with you. However, as always, centralized systems, while ease management and scalability, everything becomes a trust issue and a single point of failure or source of problems...
I can think of two organisations which could probably take care of a good chunk of the problem, if people were prepared to leave it up to them. The routing system is already largely dependent on the interoperability of bugs produced by these people, and so arguably no additional trust would be required. One organisation has a name starting with "j", and the other starts with "c". Joe
On Mon, 10 Mar 2003, Mark Segal wrote:
What surprises me most about this entire thread is the lack of centralized filtering.
Central as in 'ALL INTERNET USES MY FILTERING SERVICE' or... 'My network uses my filter service and your network uses yours'?
Since most service providers should be thinking about a sink hole network for security auditing (and backscatter), why not have ONE place where you advertise all unreachable, or better yet -- a default (ie everything NOT learned through BGP peers), and just forward the packets to a bit bucket..
This can be VERY dangerous, the default part atleast. At one point we, as an experiment in stupidity (it turns out) announced 0/1 (almost default). We quickly recieved well over 600kpps to that announcement. This in a very steady stream... When one announces a very large block like this there are always unintended consequences :( There is alot of traffic spewed out to non-available address space, this traffic is very large when aggregated :)
Which is better than an access list since, now we are forwarding packets instead of sending them to a CPU to increase router load.
Yes, routes to null0 or to a dead interface/collection host are much nicer than acls. So, for this perhaps instead of acls uRPF would be a solution for the implementor?
I don't think ARIN can help the situation. ISPs just need to remove the access lists from each router in the network and centralize them.
Or, have an 'automated' manner to deploy/audit/change said acls? RAT perhaps or some other 'automated' router config checking/deployment tool?
Regards, mark
-- Mark Segal Director, Data Services Futureway Communications Inc. Tel: (905)326-1570
-----Original Message----- From: E.B. Dreger [mailto:eddy+public+spam@noc.everquick.net] Sent: March 10, 2003 10:17 AM To: nanog@merit.edu Subject: Re: 69/8...this sucks
Date: Mon, 10 Mar 2003 09:46:33 +0000 From: Michael.Dillon
I have suggested that ARIN should set up an LDAP server to publish the delegation of all their IP address space updated
Not bad, but will the lazy ISPs set up an LDAP server to track changes they aren't tracking now? Will those with erroneous filters magically change simply because of LDAP? I still contend the answer is is a boot to the head that screams to them, "Update your freaking filters!"
Eddy -- Brotsman & Dreger, Inc. - EverQuick Internet Division Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 (785) 865-5885 Lawrence and [inter]national Phone: +1 (316) 794-8922 Wichita
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature.
These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
CLM> Date: Mon, 10 Mar 2003 17:30:27 +0000 (GMT) CLM> From: Christopher L. Morrow CLM> This can be VERY dangerous, the default part atleast. At one CLM> point we, as an experiment in stupidity (it turns out) CLM> announced 0/1 (almost default). We quickly recieved well CLM> over 600kpps to that announcement. This in a very steady Announced via IGP or BGP? I hope/assume the former, but am somewhat surprised at the traffic volume... even for UUNet. Eddy -- Brotsman & Dreger, Inc. - EverQuick Internet Division Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 (785) 865-5885 Lawrence and [inter]national Phone: +1 (316) 794-8922 Wichita ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature. These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
On Mon, 10 Mar 2003, E.B. Dreger wrote:
CLM> Date: Mon, 10 Mar 2003 17:30:27 +0000 (GMT) CLM> From: Christopher L. Morrow
CLM> This can be VERY dangerous, the default part atleast. At one CLM> point we, as an experiment in stupidity (it turns out) CLM> announced 0/1 (almost default). We quickly recieved well CLM> over 600kpps to that announcement. This in a very steady
Announced via IGP or BGP? I hope/assume the former, but am somewhat surprised at the traffic volume... even for UUNet.
bgp, no-export.
Eddy -- Brotsman & Dreger, Inc. - EverQuick Internet Division Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 (785) 865-5885 Lawrence and [inter]national Phone: +1 (316) 794-8922 Wichita
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature.
These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
CLM> From: Christopher L. Morrow
CLM> This can be VERY dangerous, the default part atleast. At one CLM> point we, as an experiment in stupidity (it turns out) CLM> announced 0/1 (almost default). We quickly recieved well CLM> over 600kpps to that announcement. This in a very steady
Announced via IGP or BGP? I hope/assume the former, but am somewhat surprised at the traffic volume... even for UUNet.
I'm not surprised. My experience with defaults in ISPs is the same. The router advertising the default (or any large prefix) becomes a "packet vacuum" for any spoofed source packet returning backscatter and all those other auto-bots and worms looking for vulnerable machines. It turns the router into a sink hole. What saves many providers today is that these large route injections are spread across all their peering routers. This is like anycasting the prefix advertisements. People are discussing is putting these advertisements on anycasted Sink Holes. So instead of having the CIDR prefixes and the Null 0 lock-ups on the peering routers, you would put them on anycast Sink Hole routers. The anycast spreads the packet black hole load over several sink holes spread over the network. Barry
BRG> Date: Mon, 10 Mar 2003 11:17:55 -0800 BRG> From: Barry Raveendran Greene BRG> EBD> Announced via IGP or BGP? I hope/assume the former, BRG> EBD> but am somewhat surprised at the traffic volume... even BRG> EBD> for UUNet. BRG> I'm not surprised. My experience with defaults in ISPs is BRG> the same. The router advertising the default (or any large BRG> prefix) becomes a "packet vacuum" for any spoofed source BRG> packet returning backscatter and all those other auto-bots BRG> and worms looking for vulnerable machines. It turns the BRG> router into a sink hole. Assuming one's upstreams and peers lack 'deny le 7'. BRG> What saves many providers today is that these large route BRG> injections are spread across all their peering routers. This BRG> is like anycasting the prefix advertisements. People are BRG> discussing is putting these advertisements on anycasted Sink BRG> Holes. So instead of having the CIDR prefixes and the Null 0 BRG> lock-ups on the peering routers, you would put them on BRG> anycast Sink Hole routers. The anycast spreads the packet BRG> black hole load over several sink holes spread over the BRG> network. IMHO, this is a good thing. Eddy -- Brotsman & Dreger, Inc. - EverQuick Internet Division Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 (785) 865-5885 Lawrence and [inter]national Phone: +1 (316) 794-8922 Wichita ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature. These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
On Mon, Mar 10, 2003 at 08:28:23PM +0000, E.B. Dreger wrote:
Assuming one's upstreams and peers lack 'deny le 7'.
Can you point out where the rule is written that noone is to announce a prefix with length le 7? Just we don't see it now doesn't mean we won't see it sometime in the future... Regards, Daniel
DR> Date: Mon, 10 Mar 2003 23:10:35 +0100 DR> From: Daniel Roesen DR> Can you point out where the rule is written that noone is to DR> announce a prefix with length le 7? Just we don't see it now DR> doesn't mean we won't see it sometime in the future... Ditto ge 25. I might have missed the RFC where that was specified; AFAIK, it's a de facto standard. Here's a big difference: Assume all /8 (except for 0/8, 127/8, and 224/3) could be aggregated. How many announcements would be saved? I could live with 200-some /8 announcements as a result of shorter prefixes being deaggregated. I suspect announcing uebershort prefixes isn't a big concern. Let's first address the issue of stray /24 prefixes. Your question is interesting in theory, but has little applicability to operational practices. It shouldn't be forgotten, and anyone using an "le 7" filter should stay on top of things... but I don't see it as a pressing issue. Better yet, let RIRs allocate based on prefix length. Then Verio-style filters would work great, save for small multihomed networks. However, if said multihomed nets used IRRs... Uhoh. Combining a handful on NANOG threads probably is a dangerous thing to do. Eddy -- Brotsman & Dreger, Inc. - EverQuick Internet Division Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 (785) 865-5885 Lawrence and [inter]national Phone: +1 (316) 794-8922 Wichita ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Date: Mon, 21 May 2001 11:23:58 +0000 (GMT) From: A Trap <blacklist@brics.com> To: blacklist@brics.com Subject: Please ignore this portion of my mail signature. These last few lines are a trap for address-harvesting spambots. Do NOT send mail to <blacklist@brics.com>, or you are likely to be blocked.
From: "Mark Segal"
Since most service providers should be thinking about a sink hole network for security auditing (and backscatter), why not have ONE place where you advertise all unreachable, or better yet -- a default (ie everything NOT learned through BGP peers), and just forward the packets to a bit bucket.. Which is better than an access list since, now we are forwarding packets instead of sending them to a CPU to increase router load.
It would be nice if vendors had a variant to (in cisco terms) ip verify unicast reverse-path that would work in asymmetrical networks. If you only have a single link to the internet, the command works well, but then why would you ever run bgp for a single uplink? -Jack
participants (11)
-
Barry Raveendran Greene
-
Christopher L. Morrow
-
Daniel Roesen
-
E.B. Dreger
-
Haesu
-
Jack Bates
-
jlewis@lewis.org
-
Joe Abley
-
Joe Boyce
-
Mark Segal
-
Stephen J. Wilcox